On-device AI is cheaper and faster than cloud

As devices such as smartphones, laptops, tablets and cars become more intelligent and powerful with increased processing capabilities and sensors, the tech industry is focusing on harnessing the power of AI on the edge or on-device AI to come up with more context-aware personalised user experiences.

And while AI may have started off as something very specific to image processing, it has over the past four to five years evolved and exploded, especially with GenAI coming into play, to include a lot more use cases around text processing, voice processing and multimodal use cases, said Rajesh Narayanan, vice president of engineering at Qualcomm.

He was speaking on ₹AI on the Edge’ at the first edition of the ‘TechForward’ research seminar series launched by Indian Institute of Information Technology, Hyderabad (IIITH). The series seeks to spark regular and deep academia-industry conversations around emerging technologies.

Narayanan pointed out that on-device AI has evolved to offer advantages such as personalisation, privacy and security, and with lower energy consumption and costs, and better performance compared to AI on the cloud.

“Today, smartphones, smartwatches, devices that are a part of home automation, devices in the car and industrial IoT are equipped with plenty of cameras and sensors and are becoming a lot more powerful with the ability to collect more data. They also have more processing capacities, leading to more context aware user experience that can lead to a much better day-to-day life for all of us,” he said.

He said that for all this to become a reality, you need a very powerful AI engine, and the overall solution has to be extremely power efficient.

Personalisation is a key selling point for on-device AI as all the data that was earlier being sent to the cloud, potentially compromising privacy and security, can be done on the device. On-device AI also brings down energy consumption significantly compared to running workloads on a much bigger infrastructure like the cloud.

Performance too improves as processing on the device itself can enable low latency as opposed to sending the query to the cloud, as with devices like Alexa.

Narayanan said on-device AI is also a lot more affordable compared to the cloud. “As we transition from traditional AI to GenAI, the cost per query is actually going up and cloud economics isn’t going to work as compared to doing it on the device,” he said.

In his talk on Vision on the AI edge’, Anoop Namboodiri, who heads the Biometrics and Secure Identity Lab and is associated with the Centre for Visual Information Technology at IIITH, said computer vision is playing a key role in pushing AI, and now GenAI.