Building PocketTune — the fastest LLM config for the phone in your hand

TL;DR — PocketTune detects your phone's Arm CPU features, benchmarks a sweep of local-LLM configurations on that phone, applies the fastest one, and gives you a fully offline chat app running it. Across three real devices it found 3.88× – 5.59× faster prompt processing. The headline finding is the one we didn't want: the phone with the worse instruction set won. No spec sheet predicts this. Only running the benchmark does.

The problem: one binary, hundreds of chips

An on-device LLM app has exactly one way to ship: build a generic arm64 binary and send it to every phone. That binary uses none of the chip-specific instructions the silicon actually has, and it picks a thread count by guessing. It has to guess, because it was compiled months before it ever met your device.

But Android isn't one phone. It's hundreds of Arm SoCs that differ in ways that matter enormously to a matmul:

• Instructions — some chips have i8mm matrix-multiply extensions, some only dotprod, a few have SVE2.
• Core topology — big.LITTLE, or tri-cluster little/mid/big, with the fast cores parked at different indices on every SoC.
• Memory — 7–8 GB shared with the OS. There is no VRAM to fall back on.

Running a local LLM well isn't one problem. It's a different problem on every phone. And the structure of that problem — which build flags, how many threads, which KV-cache layout — can't be read off a datasheet. It has to be measured on the device.

That's the whole idea behind PocketTune.

What it feels like

You install the app, open the Tune tab, and press one button.

The sweep runs on your phone, not on a server, and not on my phone months ago. Thread counts × flash attention × quantized KV cache, each config benchmarked and charted as it lands. At the end, one winner, scored 65% decode / 35% prefill — decode is what a chat feels like. Tap Apply, switch to Chat, and you have an offline assistant running the fastest configuration your specific silicon supports.

Airplane mode is the demo. Nothing after the model download touches the network.

The number: 3.88× – 5.59× faster prefill

Before writing a line of app code, we built a headless harness — harness/bench.py — that cross-compiles llama.cpp into several variants, pushes each to a real phone over adb, and runs llama-bench five times per configuration with mandatory cooldowns, airplane mode, and a battery floor.

Same Llama 3.2 1B Q4_0 model. Same llama.cpp source. Same lever — Arm-aware build flags plus llama.cpp's Q4_0 weight repack. Only the silicon changes.

Phone	SoC	i8mm?	Prefill	Best threads
Google Pixel 7a	Tensor G2	no	5.59× (24.7 → 138.0 t/s)	4
Nothing Phone 2a	Dimensity 7200 Pro	yes	4.94× (20.5 → 101.4 t/s)	2
Samsung Galaxy A34 5G	Dimensity 1080	no	3.88× (18.5 → 71.7 t/s)	2

Every number traces to a committed JSON file in results/. Nothing here is an estimate.

The finding I didn't want

Read that table again, but this time read the i8mm column.

The phone with the biggest speedup — the Pixel 7a — is the one without the matrix-multiply instructions everyone credits for Arm inference speed. It beats the phone that has them.

We expected the CPU feature checklist to predict the ranking. It doesn't. The Pixel's Cortex-X1 cores extract more from ordinary dot-product code than the 2a's A715s extract from the fancy instructions. A reasonable engineer reading the spec sheets alone would have predicted this backwards.

The thread count betrays you the same way. The Pixel wants 4 threads because it has exactly four fast cores (2× X1 + 2× A78); a fifth thread spills onto a little A55 and prefill drops 29%. The two big.LITTLE phones want 2. And on the Galaxy A34, the best thread count changes with the build — 6 threads for the generic binary, 2 once the arch flags are on. Same phone, same day, different answer.

That is the entire thesis in one table: you cannot read your phone's fastest configuration off a spec sheet. So PocketTune doesn't ship a lookup table. It ships the measurement loop, and runs it on the phone in your hand.

The lever that came out flat — published anyway

We also tested KleidiAI, Arm's optimized microkernel library, fully expecting another jump on top of the arch-flag gains. On all three phones, for Q4_0, it landed within noise of the plain arch-flags build:

Phone	KleidiAI vs arch flags
Nothing Phone 2a	≈ 0%
Galaxy A34 5G	+0.03%
Pixel 7a	137.95 vs 137.95 t/s — identical to four decimals

llama.cpp's own aarch64 repack path already exploits dotprod/i8mm well for this quantization, so the gains are attributable to arch-aware codegen plus repacking — not to any one kernel library.

This is the result I least wanted, and it's in the README anyway. A project whose entire claim is "measure, don't assume" doesn't get to quietly drop the lever that came out flat — three times, on three SoCs, from two vendors. It stays in the sweep, because the finding is specific to Q4_0 on these chips, and other quantizations may well disagree. Burying it would make every other number in the project less trustworthy.

How it works, in one breath

Everything a user touches is TypeScript. Everything fast is prebuilt native code selected by runtime CPU detection. No C++ or Kotlin was written for this project.

The Tune tab is one loop:

1. Detect — cpu.ts reads /proc/cpuinfo and the cpufreq topology: feature flags, core clusters, max clocks, which cores are the big ones.
2. Plan — tuner.ts builds the sweep from what it found. A tri-cluster chip gets different thread candidates than a big.LITTLE one.
3. Bench — each config runs through llama.rn's bench(), with the battery rails sampled where the kernel exposes them.
4. Score — 65% decode, 35% prefill.
5. Apply — the winner is persisted.
6. Chat — offline, with measured tok/s on every reply.

The engine wrapper holds one llama context at a time — a phone doesn't have RAM for two — and serializes every native call, so switching tabs mid-benchmark can't interleave with a measurement.

llama.rn ships six arm64 kernel builds and picks one at startup by CPU feature: an i8mm-capable chip gets v8_2_dotprod_i8mm, a dotprod-only one gets v8_2_dotprod. That runtime dispatch is the app-side equivalent of the explicit per-arch builds the harness compares.

Five tabs

Tab	What happens there
Device	The dotprod / i8mm / SVE2 / SME2 checklist, the big.LITTLE core map with clocks, and which of llama.rn's six kernels the runtime dispatch selected for this chip
Models	Downloads a GGUF (Llama 3.2 1B Q4_0, ~700 MB) or picks up one you pushed over adb
Tune	The sweep — threads × flash attention × quantized KV cache — charted live, winner applied in one tap
Chat	A fully offline assistant running the applied config, measured tok/s on every reply
Lab	The published cross-phone evidence, plus this phone's own tuning history — every number traceable to raw JSON

Where the kernel exposes the battery rails, the sweep also reports tokens per joule — because on a battery-powered device, fast and efficient are not always the same config.

Reproduce every number yourself

The published results come from llama-bench builds driven over adb. No app involved, so anyone can verify them:

# on a phone WITH i8mm (e.g. Nothing Phone 2a)
python harness/bench.py --model models/Llama-3.2-1B-Instruct-Q4_0.gguf \
  --variants generic arch kleidiai

# on a dotprod-only phone (e.g. Galaxy A34) — the i8mm builds SIGILL here
python harness/bench.py --model models/Llama-3.2-1B-Instruct-Q4_0.gguf \
  --variants generic dp-arch dp-kleidiai

Median of 5 runs, fixed prompt and generation lengths, 90-second cooldowns between variants, airplane mode, screen forced awake, battery level and temperature recorded before and after.

And the variance, stated plainly: the Pixel 7a's generic baseline is noisy — it moved ±9% across repeat runs, and one cold-phone run read ~25% high before settling. Its arch builds repeat to within 0.3%. The published Pixel figure is the conservative run, so 5.59× is if anything an understatement; the flattering run would have read 6.10×. Both are committed. Nothing is dropped for being inconvenient.

Try it

git clone https://github.com/ayanbag/PocketTune
cd PocketTune/app
npm install
npm run android

Then: Tune → download a model → run the sweep → Apply → Chat.

Adding your own phone to the published set takes one command and no code — the harness detects the chip, picks its build variants, and writes a results/<device>-<timestamp>.json, which is the same unit the app's Lab tab and the project site consume.

OR

You can just download the app from github.com/PocketTune, which is a fully offline chat app running the fastest configuration for your specific silicon.

At last

PocketTune is MIT-licensed. Source and raw benchmark data live at github.com/ayanbag/PocketTune. Built for the Arm Create: AI Optimization Challenge 2026, Mobile AI track.

---

Related Links

github.com

github/PocketTune

The source code, benchmark harness, and every raw result JSON behind the numbers in this post.

ayanbag.github.io

PocketTune

The project site — device panels, the attribution ladder, and the thread-scaling curves for every phone measured.