On Device AI for Mobile Apps

Why run AI on the device

Most AI features call a model in the cloud. That works well until the user has no signal, the data is too sensitive to send, or the feature needs to respond faster than a network round trip allows. Running the model on the phone itself addresses all three, and current iPhones and Android flagships include dedicated neural processors built for exactly this work.

Tepia recommends on device AI when one of these is true:

• Privacy is the product: a journaling, mental wellness or health app where users will not accept their entries leaving the phone.
• The network is unreliable: field crews in crawlspaces, mines, hospitals or rural areas who still need recognition or transcription.
• Speed is visible: live camera features, keyboard suggestions or audio processing where any lag feels broken.
• Volume is high: a feature used dozens of times a day per user, where server calls add load that grows with every new install.

When none of these apply, a cloud model is usually simpler and more capable. Tepia will tell you that in Discovery rather than build on device AI for its own sake.

On device or cloud: a decision guide

Tepia walks through these factors with you for each AI feature, because the answer often differs within the same app:

Most production apps Tepia builds end up hybrid. A focused task like classifying a photo, transcribing a short memo or suggesting a reply runs locally, while long summaries, complex questions or anything needing your company’s knowledge base goes to the cloud. The app decides per request and tells the user when content will leave the device.

Frameworks and models Tepia works with

The on device toolkit has matured quickly, and the right choice depends on platform and task. Tepia evaluates these options in Discovery.

iOS

Core ML runs custom and converted models on the Neural Engine, GPU or CPU. Apple’s Vision and Speech frameworks cover text recognition, image analysis and transcription without a custom model. On devices that support Apple Intelligence, the Foundation Models framework gives apps access to Apple’s on device language model for summarizing, extracting and generating short text, with no data sent to a server.

Android

ML Kit provides ready made text recognition, barcode scanning, face detection and translation. Gemini Nano, available on supported devices through Android’s AICore system service, handles summarization and rewriting locally. LiteRT, formerly TensorFlow Lite, and ONNX Runtime run custom models across a wider range of hardware.

Cross platform

For React Native and Flutter apps, Tepia writes native modules around these frameworks so each platform uses its own optimized path while the shared app code stays the same. Our offline first app development page covers the sync side of apps that must work without a connection.

Engineering constraints to plan for

On device AI introduces constraints cloud features never face, and Tepia plans for each during Design:

• Model size: quantizing a model to lower precision shrinks it several times over with modest accuracy loss. Larger models download after install rather than shipping in the bundle, so the initial app stays small.
• Device spread: an older Android phone may lack the memory or accelerator a model needs. Tepia detects capability at runtime and falls back to a smaller model or the cloud.
• Battery and heat: continuous camera or audio inference drains batteries and warms phones, so Tepia throttles frame rates and batches work.
• Model updates: improving a bundled model means an app release, so Tepia often ships models as versioned downloads that can be updated and rolled back.
• Evaluation: the compressed model is tested against the same examples as the original, on real devices, because accuracy can shift after conversion.

These details decide whether the feature feels instant on a three year old phone or only on the newest flagship. For camera specific work, see AI image recognition in mobile apps.

How Tepia approaches on device AI

Tepia’s six phase process adapts to the hardware realities of on device work, with a US based project manager coordinating design and engineering.

1. Discovery: a review of your users’ devices from analytics, the privacy and offline requirements, and a benchmark of candidate models on real phones. Deliverables are an Investigation Summary with the on device versus cloud recommendation, an Interview Summary and User Stories such as a user summarizing a private note in airplane mode.
2. Design: wireframes for model download progress, offline states and clear notices when data goes to the cloud.
3. Development and Testing: model conversion and quantization alongside app development, with an Alpha on a device lab covering old and new phones and a Beta in the field. Test plans include accuracy after conversion, battery drain, heat and response time.
4. Training: sessions for your team on model versioning, rollout controls and reading device performance reports.
5. Launch: App Store and Google Play submission, staged model downloads and monitoring of crashes and slowdowns by device.
6. Support: model updates, compatibility work for each new OS release, and issue resolution.

Discovery generally runs 1 to 3 months and development and QA 2 to 6 months. See our process or contact Tepia.

Frequently asked questions