Your laptop can run the model. Running the agent is a different question.

Nearly every local-LLM benchmark reports tokens per second at an empty context. Send a short prompt to a freshly loaded model, read the number off the end. It is easy to reproduce and it is what the tools print by default. For a coding agent it is close to useless.

A coding agent never sees an empty context. It opens with a system prompt, tool schemas, a few files, a transcript. By the time it generates its first token it is thousands of tokens deep, and it stays there for the rest of the session. If you picked your hardware on a number measured at depth zero, you measured a workload you will never run.

I spent a few weeks measuring this on one machine. Two results surprised me.

The agent's own prompt sets the hardware floor

Start with the dull one, because everything else sits on top of it.

The CLI I work on has a system prompt measuring 9,548 tokens. Add the smallest reply it will ever ask for, 1,024 tokens, and a single turn cannot happen in a window under 10,572. That is not a window where the agent works well. That is the window where the prompt fits and one minimal answer fits and nothing else does. Not your question, not a file, not a tool result.

Round up to the next power of two and you get 16,384, which is the smallest window worth sizing a machine for. Our floor used to be 8,192, which sits below the prompt it existed to make room for.

Why did that survive? Because under the floor nothing errors. llama.cpp reserves your requested max_tokens out of n_ctx before it tokenizes anything, so as the prompt grows the space left for the reply shrinks. Push far enough and the model gets one token to answer in. A one-token reply is a valid reply. No warning, no failure, just an agent that has become inexplicably stupid.

If you are debugging a local setup that runs but seems dim, check the window before you blame the model.

The engine ranking inverts with depth

Here is the one I did not see coming. Measured on a Snapdragon X Elite with a 4B model, across the three engines that machine offers: CPU, Adreno GPU, and the Hexagon NPU.

At depth zero the CPU wins. Roughly 22 to 30 tokens per second of decode, against about 19.7 on the GPU and 18.5 on the NPU. Benchmark the way everyone benchmarks and you conclude the accelerators are not worth the trouble.

At depth 469 the order flips. NPU holds 18.55 and the GPU holds 18.05, while the CPU has dropped to 13.2. By the time the context is realistically full the CPU has given up about 42 percent of its decode speed. The GPU gives up 8 percent. The NPU is flat.

The obvious explanation is wrong. This is not the KV cache eating bandwidth: the cache for that model runs about 144 KiB per token, so 469 tokens of context adds under 3 percent to the bytes moved per decode step. Three percent more data does not cost you 42 percent of your throughput. Whatever the CPU is losing, it is losing to something else, and I do not have a clean account of what.

So a one-line benchmark prompt ranks your hardware in reverse order from the workload you actually have. For a chatbot those CPU numbers are honest. For an agent they are a trap.

There is a companion effect that bites the same way. Run an unrelated job taking about 20 percent of your CPU and measure again: the NPU loses 1.2 percent, the GPU loses 64. The GPU path dispatches from the host once per token, so anything competing for cores lands directly on decode. The Hexagon has its own clock domain and mostly ignores you. Which means a benchmark run on an idle machine describes a machine you do not have, because you are going to be running a compiler at the same time.

On the NPU, the window is a throughput knob

The last result is specific to the NPU path, and it is invisible in every field you can inspect.

Genie's KV tensors are graph inputs, statically shaped to the window the bundle was compiled for. The whole buffer streams on every decode step whether or not the positions in it hold anything. The window is not a memory setting you pay for when you fill it. It is a throughput setting you pay for always.

Same model, same machine, three bundles:

compiled window	decode
4,096	18.0 tok/s
8,192	8.8 tok/s
16,384	3.3 tok/s

Decode is flat with respect to how full the window actually is. A 16k bundle holding 469 tokens runs at 3.26. Holding 10,532 it runs at 3.27. You are paying for what you compiled.

None of this appears in the bundle's own metadata, and it does not appear at the server's /props endpoint either. Two bundles compiled with different window strategies can report an identical KV shape.

The fix is to export with several context lengths so the runtime picks the smallest graph that fits the request. On our numbers that bought 2.98x on prefill and 2.07x on decode for 3.8 percent more disk, with no additional memory on the accelerator.

What this is worth

One machine. A Snapdragon X Elite on Windows ARM64, with sample sizes between one and three. The depth inversion is the result I would most expect to generalize, since it is about how different processors handle growing context rather than about this particular silicon, but I have not measured it anywhere else and you should not assume it either.

Every number above comes out of snapdragon-genie-server, which is public and carries the harnesses that produced them along with the conditions each was taken under. It also carries the ones that did not survive: a batch of figures deleted for want of evidence, and a concurrency result retracted when a controlled rerun disagreed with it. If you are going to act on a benchmark, the retractions tell you more about it than the headline does.

It is also slow. A real turn through our CLI on that machine takes about three and a half minutes, and almost all of it is prompt evaluation rather than generation. Local inference for a coding agent buys you privacy and a zero bill, and charges you latency. Whether that is a good trade depends entirely on what you are doing, and for plenty of work it is not.

The part I would keep regardless of hardware is the measurement discipline. Benchmark at the depth you will actually run at. Check what your window costs you before you go looking for a faster model. And be suspicious of any number that was collected on an idle machine, because you will not be running one.

The CLI in question is typed, whose default tier works this way.