Kev reads a document once and answers every question about it separately — Question-isolation masks — What does it mean?
The news. On September 20, 2026, the Kev project published three open decision models — Kev-0.8B, Kev-4B and Kev-9B — each a rank-16 LoRA adapter plus a pointer head on a frozen Qwen3.5 base, under Apache-2.0. They take typed questions and return probabilities in one forward pass, with no text generated — though the project notes they are not well calibrated on sources it was not trained on. On the locked out-of-domain test the project reports 0.837 accuracy for Kev-9B, 0.832 for Kev-4B and 0.668 for Kev-0.8B. The release also records what moving to Qwen3.5 cost: the new bases mix recurrent layers into the stack, and those layers change how questions have to be kept apart. Read the release →
Picture the briefing room. One board at the front carries the whole document — a support ticket, a refund policy, a contract. Five contestants sit in booths facing it. Every one of them can read the board; none of them can hear the others. Ask each booth a different question and you get five answers that are about the same document and about nothing else.
That room is the layout Kev feeds to an attention-only model — the Qwen3 bases the project shipped first. The document — the project calls it the state — goes into the token sequence once, and each question is appended after it as its own block of tokens, ending in a <decide> marker. What makes the booths soundproof is the attention mask: a token may read the state and its own question, and nothing else. The mask you already know from text generation blocks the future; this one also blocks sideways.
One more detail keeps the booths honest. Each question's position IDs restart just after the state, so the fourth question in the packed sequence is numbered as though it were the first. It cannot work out from its own position that other questions are sitting in front of it. The model then does what the room implies: it works through the document once and answers each question independently.
Naive: One at a Time
Step 1 — only R1 runs
Step 2 — only R2 runs
Step 3 — only R3 runs
GPU reads weights 3× for 3 requests
Engine: All at Once
Step 1 — all 5 run together
GPU reads weights 1× for all 5 requests
Gray = idle GPU capacity. Batching fills the GPU on every step.
So far the booth walls are made of attention. That is exactly the catch, because an attention mask is a rule about attention, and nothing else in the model is obliged to follow it.
Qwen3.5 — the base this release moved to — is not an attention-only stack. It interleaves attention layers with Gated DeltaNet layers, which are recurrent: rather than looking back over a table of every earlier token, they carry a running state forward, one token at a time. There is no table of earlier tokens to mask, so the mask simply does not apply. A recurrent layer that has already processed question 1 cannot be instructed to unsee it — question 1 is folded into the state it hands to question 2. Put the two designs side by side and the booth walls only exist in one of them.
Kev's answer is to stop packing. On a hybrid base each question runs as its own row — the state, then that one question — and the server computes the state's cache once and reuses it for every row. Isolation stops being a rule the mask enforces and becomes a property of the layout: the rows never touch. The metaphor survives the change, it just restages. Instead of one room with soundproof walls, you photocopy the briefing and walk each contestant into a room of their own.
The project checks that the two stagings agree rather than assuming it. On attention-only bases, asking the questions packed together and asking them separately produce probabilities within 4e-6 of each other in fp32 tests. It is also careful about what that does not mean: isolation holds between questions, not inside one. Reordering the options within a single question can still change that question's answer.
| Form | Backbone | What keeps questions apart | What it costs | Source |
|---|---|---|---|---|
| All questions packed into one sequence | Attention-only (Qwen3 bases) | The attention mask — read the state and your own question, nothing else | One forward pass for the whole batch | Kev README |
| One row per question | Hybrid attention + Gated DeltaNet (Qwen3.5 bases) | The layout — separate rows never meet, so isolation is exact | One row per question, over a state cache computed once and reused | Kev README |
| Agreement between the two | Attention-only, fp32 parity tests | — | Probabilities differ by at most 4e-6 | Kev README |
Hold the request shape fixed at the one the project benchmarks: five questions, three options each, over a ~230-token state. Count only the state tokens: this is arithmetic on document processing alone, it leaves out the question blocks, and it is not a prediction of runtime.
Ask those five questions as five separate requests with no cache reuse and the model works through the state five times: 5 × ~230 = ~1,150 state tokens. Share the state instead — packed behind one mask, or as five rows over one cached state — and the document is read once: ~230 state tokens, plus the five short question blocks. On the state, that is ~1,150 → ~230 tokens, a ~5× cut, and the ratio grows with the number of questions — ten questions about the same ticket would be ~2,300 → ~230 — while the absolute tokens saved grow with the document's length. (These token counts are arithmetic on the benchmark's stated shape, not a measured figure.)
The project's own measurement of state reuse points the same way at a different layer. With a repeated 772-token state, the server's prefix cache lets Kev-4B on the Qwen3 base answer in 242 ms instead of 861 ms — the same idea, applied across requests rather than across rows, and measured rather than derived.
Goes deeper in: LLM Internals → Attention → Causal Masking
Related explainers
- vLLM v0.20 — FlashAttention 4 packing — the other reason to pack many sequences into one attention call: the kernel stops chewing through padding.
- Training-free linear attention — what else changes when a model's attention is swapped for a fixed-size recurrent state.
- Gated DeltaNet 2 — inside the recurrent layer type that ignores the mask.