Edge Impulse: On the rise and rise of embedded and edge ML

James has a passion for how technologies influence business and has several Mobile World Congress events under his belt. James has interviewed a variety of leading figures in his career, from former Mafia boss Michael Franzese, to Steve Wozniak, and Jean Michel Jarre. James can be found tweeting at @James_T_Bourne.

When ChatGPT was launched, and the seemingly limitless potential of large language models (LLMs) became apparent, enterprises across industry wanted a piece of the action. The problem was: how could an industrial manufacturer, for example, truly integrate this potentially game-changing technology into their processes and machinery?

The answer lies in the application of embedded and edge machine learning (ML). “The core concepts of machine learning are much easier to get to market – it’s much less risky than perhaps people would think,” explains Jim Bruges, solutions engineer at Edge Impulse, provider of development tools for ML on edge devices. 

The platform and wider tools are end-to-end: Edge Impulse is able to collect and label high quality sensor data, and develop ML and signal processing algorithms to deploy across any device, from the smallest of microcontrollers to larger AI accelerators. It is important to emphasise that while the company is described as a platform provider, the ability to collect data at scale is just as important. 

There are various advantages to being able to run ML workloads in the smallest spaces, from getting data where bandwidth constraints would have previously been a no-go, to avoiding the cost and latency concerns of processing data in the cloud. The result, for product leaders at forward-looking organisations, is an extra layer of edge intelligence with cleaner, more insightful data.

The AI hype train, however, can travel in both directions. Amid all the articles showcasing how generative AI will turn industries upside down, there are nearly as many warning of hallucinations, biased outputs, and more. Given how traditionally risk-averse enterprises are, this is not an insignificant factor. Yet edge ML can help mitigate these concerns. 

“There is an element of risk around large language models with hallucinations, but if you can control the data going into these machine learning models, and you can limit the scope of them to something as simple as detecting faults in your product, or detecting someone’s usage of a product, then you can make a much more robust and smaller model, that then goes on to those devices and adds value to the customer in a more meaningful way,” says Bruges.

This is not the only misconception with AI, either. “One of the dangers of the current wave of artificial intelligence is that it has this aura of being incredibly complex and not understandable,” says Bruges. “When it gets things wrong, people lose trust in it quickly.

“As an engineer myself, if something goes wrong, I want to work out why,” he adds. “If you’ve got a large language model with 7.5 billion parameters in it, you’re never going to find the pattern that caused it to hallucinate. What you can do is inspect the data at a large scale that that model ingests.

“When you start to look at machine learning at a smaller scale, it actually does become simpler to understand.”

For Edge Impulse, the proof of the pudding is in the eating. With industrial and healthcare clients trusting the platform, those examples are ‘not something where you want to let loose a model that’s going to hallucinate’, as Bruges puts it.  

Whatever the industry, the classic use case involves a customer whose product has a microcontroller which they want to make smarter. A hypothetical example would be a company in the HVAC (heating, ventilation, and air conditioning) industry looking to create an air conditioning unit with a smart sensor which detects whether humans are present in the room, and then heat or cool appropriately. This ties into Bruges’ earlier project definition: a clear objective with limited, controllable scope. 

The primary point of contact for many of the companies who work with Edge Impulse is the R&D department, or the head of innovation. This is certainly understandable, for early-stage projects at the bleeding edge of digital transformation. But what about the next step: moving to production? Again this can be assuaged. As the results come, the project moves through the company. 

“That’s important to customers – understanding that, if you need to switch hardware for production, you can have the confidence that our platform will be able to compile this model down for any hardware, as long as it has the capability to run the model at the processor level,” says Bruges.

“At the research and development stage, we come in and we provide [the customer] the tools to iterate quickly, and experiment, and provide real value to their research department,” adds Bruges. “Then as that relationship develops through to production, we may get handed from the R&D team over to the production development team.

“That’s where our expertise around data collection at scale, clinical trials, and running data collection in a robust, auditable way becomes really important,” says Bruges. “You need to have the confidence that the data going into these models is going to deliver a robust and reliable output at the other end, and the only way you can do that is by collecting data in a methodical manner.”

As anyone who is responsible for digital transformation will know, projects only work when everyone is bought in. Edge Impulse tailors its offering from product leaders, to AI practitioners, to embedded engineers; but getting product owners and engineers to speak the same language is a different challenge altogether.

“A lot of the groundwork of engineering is actually about communicating across disciplines,” explains Bruges. “It’s not necessarily about the kind of hardcore maths and physics that are involved in developing a product; it’s about bridging the gap between teams and allowing collaboration by having a common language of understanding.

“I think that’s something that we work quite carefully at Edge Impulse to do,” continues Bruges. “Not just on the product side where we develop this tool that can be used by anyone, but also in the way we talk to customers and try to bridge the gap between data science teams and embedded teams, and product leaders, and directors of companies, because we need everyone to understand the impact of everyone else’s work in that workflow to get to the very end.”

As befitting its edge AI ethos, Edge Impulse is participating at TechEx Global in London on November 30-December 1 in both the AI & Big Data Expo and Edge Computing Expo events. Bruges is speaking on a panel which explores emerging technology convergence; an area which is a natural fit. 

“I think there will be an interesting discussion trying to work out where this new state of tools – whether we’re talking about large language models, talking about machine learning for the edge, all of this stuff – fits into a 21st century company, and then where the real innovation is happening, and where companies can get an actionable edge on their competition,” says Bruges. “There is so much opportunity for edge machine learning to add value to all of these companies – and that’s what I want these product leaders to think about.”

Photo by Monstera Production

Want to learn more about edge computing from industry leaders? Check out Edge Computing Expo taking place in Amsterdam, California and London. 

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: , ,

View Comments
Leave a comment

Leave a Reply