Delivering AI projects for our clients often means taking machine learning out of the data centre and into the real world.  

The AI renaissance: why it’s taken off and where it’s going

Out here, the constraints of size, cost, power and connectivity present significant challenges in getting Edge AI to work at its best.

This was the backdrop to a recent project we completed for an Asian technology giant. 

The project resulted in accelerating our client’s AI algorithms by nearly 10x on candidate edge compute devices. This also gave them a true performance benchmark on which to base their future design decisions. 

In my video I share more insights from the project, which leveraged a unique combination of expertise across our AI and semiconductor groups. This expertise has generated billions of dollars of value for our clients by creating and optimising world-leading silicon platforms.

If you’d like to talk more about Edge AI or any other business challenge needing a unique AI solution, please get in touch at tim.ensor@cambridgeconsultants.com  

You can find more about our work in AI and analytics here

 

Transcript 

We’re seeing artificial intelligence systems equalling or bettering human performance on tasks almost every week. To really capture the value of these achievements though, our clients are working through the challenge of how get these algorithms out of the data centre and onto edge devices in the hands of their customers. 

Our team recently helped the AI development team at an Asian technology giant with the edge component of their intelligent platform. Our work involved accelerating their AI algorithms by nearly 10x on candidate edge compute devices. This gave them a true performance benchmark on which to base their design decisions. 

The challenge they were facing was that, whilst AI frameworks like Tensorflow and Pytorch are open and cross-platform, as soon as we deploy models to a specific processor, the tools become much less straightforward and are specific to each device. Consequently, our client engineering team had struggled to get some of their models to run fast enough and others failed to convert at all. 

Our team was able to identify parts of each model which were not needed for inference, or operations which were supported in different ways on the different processor platforms. By considering these and a host of other optimisation techniques, we shrank the model by 75%, the memory use by 80% and made it run 10x faster. All whilst only dropping 3% in accuracy. Where certain key features of other models were simply not supported by the tools, we could advise on potential workarounds. 

As a result of our work, the client had a true measure of the performance of their candidate processor platforms for specific workloads and an understanding of the abilities and constraints of the tools. This gave them confidence to move forwards with their vision of putting AI into the hands of their customers in factories, hospitals and cities globally. 

If you’d like to talk more about building high-performing embedded AI systems at the edge or any other business challenge needing a unique AI solution, please get in touch. 

You can find more about our work at cambridgeconsultants.com/ai 

Author
Tim Ensor
Commercial Director

Tim works with clients across many sectors to help them achieve business impact with world-changing technology innovation. Tim has had a string of commercial leadership roles focused on innovation in fields including telecoms, logistics and energy and working with world-leading AI, robotics and connectivity technology. He's an electronic engineer, Cambridge MBA and optimistic about using technology to make the world better.

Connect on LinkedIn