For the past two years, on-device AI in mobile apps was the category people discussed at conferences and shipped in pilots. In 2026, it has crossed the line into routine production work.
The trigger is a latency milestone: sub-20ms inference for production computer vision models on mid-range devices — specifically 2024–2025 chipsets with standard NPU and GPU acceleration. That performance is now reproducible on phones that cost £400. It’s no longer a high-end hardware story.
What’s driving the shift from pilot to production
Four forces have converged this year:
Privacy by architecture. When computation happens on-device, data never leaves the handset. For applications that process faces, medical images, or financial documents, this isn’t just a privacy preference — it’s increasingly a legal requirement under UK GDPR, NHS data governance rules, and EU AI Act provisions for high-risk applications. On-device inference lets you eliminate a whole category of compliance paperwork.
Framework maturity. Meta’s ExecuTorch 1.0, released in late 2024, has become the standard for edge AI deployment on iOS and Android. Google’s LiteRT (the rebrand of TFLite) has graduated into a full production stack with 1.4x better GPU performance than the previous generation and new NPU acceleration paths. These aren’t experimental SDKs — they’re production infrastructure used by large-scale consumer apps.
Smaller models that work. Where 7B parameters once seemed the minimum for useful language model outputs, sub-billion models now handle many real tasks. Llama 3.2 (1B/3B), Gemma 3 (down to 270M), and Qwen2.5 (0.5B–1.5B) are all deployable on mid-range devices with acceptable quality for classification, structured extraction, and short-form generation.
Connectivity independence. Apps that rely on cloud inference are dependent on a connection. Offline-first capabilities matter in healthcare, field service, and logistics — sectors where BuildApps works regularly.
What this means for product teams commissioning apps in 2026
On-device AI is worth evaluating at the start of a build, not as an afterthought. The questions to ask: does this application process sensitive data that shouldn’t leave the device? Does it need to work offline or in low-connectivity environments? Is sub-20ms response time important for the interaction model?
If yes to any of these, cloud inference might be the wrong default. On-device is no longer the harder choice — in some builds it is now the simpler one.
The healthcare and fintech verticals are moving fastest. Any mobile product in those sectors being specified now should have on-device inference in the initial architecture conversation.
More on our approach to AI features in mobile apps: AI-assisted development and healthcare software development.