Cactus ships Needle 3 as 2-to-20-layer subnetworks — Laddered Simple Attention Networks — What does it mean?

The news. On September 19, 2026, Cactus released Needle 3, a family of on-device models for tool calling, structured extraction and embeddings, aimed at phones, wearables, browsers and embedded boards. The headline is not one model but a range: 29M to 121M parameters across 2 to 20 layers, shipped as 9–29 MB 2-bit binaries, with a reported 400–4,000 decode tokens per second (generating output) and 1,000–10,000 prefill tokens per second on a Raspberry Pi 5 (reading the input prompt), behind an inference engine under 1 MB. The architecture that lets one base cover that whole range is what this explains. Read the release →

Picture the telescoping ladder again. You do not own a short ladder and a medium ladder and a tall one; you own one ladder and a decision about how far to extend it. The rungs are always the same rungs in the same order, and the two parts that do the real work — the foot plate that stops it sliding and the hook that catches the gutter — come with you at every height.

That is the shape of Needle 3. Each rung is a transformer block, and the height you extend to is the number of blocks you ship. Cactus's own architecture summary reads "ladder: any depth 2..20, blocks 0 and 19 kept" — so a short export is not simply the first few blocks; the first block and the last block ride along at every height, the way the foot and the hook do. Everything in between is what you trade away for memory.

The claim the company makes for this is unusually flat: "Every layer of Needle 3 is a sub-network with monotonically increasing capacity." Monotonic is the load-bearing word — it means capacity only ever goes up as you add rungs, so "take the tallest rung that fits" is a sound default rather than a coin flip. It is a claim about capacity, not a promise about accuracy: you still evaluate the depth you actually intend to ship.

Why depth, and not width? Because the two axes fail differently. The heads inside one attention layer run side by side, so narrowing them changes what a single layer can attend to at once. Layers run one after another, each reading what the one below it wrote, so cutting depth removes whole sequential stages of refinement rather than thinning any one of them. Needle 3 holds its width fixed — the published summary gives 768 dimensions and 12 query heads to 2 key/value heads at every rung — and moves only along depth, which is why the exports differ in layer count and not in shape. Multi-head attention is the width dial; the ladder is the depth dial.

What Cactus documents about the workflow is short and specific. Fine-tuning happens once, at full depth, against a frozen base — needle finetune data.jsonl produces a LoRA adapter, and any smaller subnetwork can be built afterwards as a .cact file — the page describes a 4-bit export on the fine-tuning path, and CQ2 2-bit binaries for the 9–29 MB range it advertises. What the page does not document is how the ladder itself is trained, or how the CQ2 format works; it gives sizes and speeds, not a recipe. Treat the ladder as a published capability here, not a reproducible method.

The other half of the release is what a sub-30 MB model is meant to do once it is on the watch. Tool descriptions and schemas constrain the JSON it emits, and every response carries a score from the calibrated confidence head. Cactus describes the policy that goes with it: at 0.7 or above the app acts at once, in the middle it shows the proposed call and asks, and an empty result is read as a refusal. On a device, a small model that knows when not to answer is worth more than a slightly larger one that always does.

Where the ladder earns its keep. Hold three things fixed: the published range (2 to 20 layers, roughly 9 MB to 29 MB), one frozen 20-layer base, and one LoRA adapter trained at full depth. Say you ship to three device tiers — a watch, a phone and a set-top box (illustrative). Under the old arrangement each tier is its own project: 3 base trainings + 3 fine-tunes = 6 training runs, each with its own data pipeline. Under the ladder you train the base once and fine-tune once, then run needle build --layers X three times: 6 training runs become 2, and the three exports are builds rather than experiments. What does not shrink is evaluation — you still test three binaries, because three depths are three different models — so the saving lands on the training side, not on QA. For how far down the ladder you can actually go, the data point Cactus offers is a 4-layer export which, after fine-tuning, passes or exceeds DeepSeek V4 Flash on DroidCall, a benchmark that scores whether a model picks the right on-device function and fills its arguments correctly, with fine-tuning reportedly worth +18 to +36 points there.

Goes deeper in: LLM Internals → The Transformer Block → Modern Variants & Scale

Related explainers

• Progressive layer dropout for depth-elastic transformers — the training-time version of the same axis: blocks switched off while training, rather than cut at export.
• KV-compatible early exit — choosing a depth per token at inference, instead of fixing one depth per device.
• DRAM-aware MoE scaling for sub-3GB devices — the same memory ceiling approached along the sparsity axis instead of the depth axis.