ReaDy is a speculative-decoding drafter for hybrid linear-attention models. Each draft position forms its own query and reads the target's committed recurrent states, then adds the result to its features through a residual update.

Why read the recurrent state instead of the KV cache?

The recurrent state has a fixed size per head (128 × 128 in the paper), so the added read does not grow with the prefix. A token-wise KV cache grows with every token, which raises the I/O cost of any extra read.

How much faster is it than DSpark?

On Qwen3.8-27B with one NVIDIA B200, ReaDy averages 302.78 tokens/s versus 280.42 for DSpark, a 7.97% gain, and 8.50% longer acceptance. Part of that comes from extra fine-tuning; against a fine-tuned control without state access the gain is 5.68%.

Does it change what the target model outputs?

No. ReaDy changes only how drafts are conditioned. The target still verifies the block and accepts the longest prefix that matches its own greedy choices, and only committed states are read.

LLM· 2026-10-16

ReaDy speeds speculative decoding 3.16× — Recurrent-state drafting — What does it mean?

Jargon

Hybrid linear attention (HLA): A model that mixes a few full-attention layers with many recurrent layers, which keep a fixed-size memory instead of a growing KV cache. Qwen3.8-27B, the paper's target, is one.
Recurrent state: The fixed-size matrix a linear-attention layer updates after every token. It summarizes the whole prefix so far; in the paper it is 128 × 128 per head at any prefix length.
Drafter and target: In speculative decoding the drafter is a small fast model that proposes several tokens and the target is the big model that checks them in one pass. Primer →
Acceptance length (τ): The average number of tokens committed per verification round, counting the token the target supplies itself. Higher τ means more progress per expensive target pass.
KV cache: The keys and values that full attention stores for every past token so it need not recompute them. It grows with every token. Primer →
DSpark: The drafter ReaDy builds on: a parallel backbone that predicts a block of tokens at once, plus a light sequential head that links the tokens inside the block.
Low-rank read: Compressing a matrix to a few directions (here rank 8 out of 128) before using it, so the read touches less data.

The news. On October 14, 2026, a research team (StartLux, with the University of Hong Kong and others) posted ReaDy (arXiv:2610.20503): a speculative-decoding drafter for hybrid linear attention models that lets every draft position retrieve its own context from the target's committed recurrent states. On Qwen3.8-27B with one NVIDIA B200 it averages a 3.16× speedup over autoregressive decoding across seven benchmarks, and 7.97% more throughput than the DSpark drafter it builds on. Read the paper →

Picture a meeting where a chair must approve a draft of the next seven lines of the minutes. The junior drafters sketch all seven lines in one pass (a clerk then links each line to the one before it), and their only help is the highlights the chair already marked, which are the target's hidden features. Those highlights cover what the chair needed for the last line, not for line five. The other option is the full minutes stack, which has every page but grows by a page every token. A hybrid linear-attention model keeps a third thing: one running summary sheet whose size never changes, and ReaDy lets each junior drafter ask that sheet its own question.

Concretely, a hybrid linear-attention model replaces most full-attention layers with recurrent layers that fold every past token into a matrix state per head, here 128 × 128 per head whatever the prefix length. ReaDy adds a small state reader to the last layer of its DSpark-based drafter, after attention and before the MLP. For each draft position it forms a query through a rank-8 bottleneck, applies it to the target's committed states from two layers (30 and 62) compressed to rank 8, and adds the result back as a residual update. Only states from committed tokens are read and states from rejected drafts are dropped, so verification is unchanged: the target still accepts the longest matching prefix, as in the verification algorithm.

The paper's theory says why a second look is not redundant. A state answers different queries differently: two states can agree on every query the target already asked and still disagree on a new one, whenever the new query points outside the directions the target's queries covered. A draft position several tokens ahead has exactly such a different need. On cost, the added channel holds M·128·128 numbers for M layer-head pairs, independent of prefix length, while a token-wise cache holds M·t·256, so by the paper's formulas the state is smaller once the prefix passes 64 tokens (computed from those formulas; real memory traffic also depends on execution and reuse). Fused gather and preparation kernels cut state-gather latency from 16.388 to 4.622 μs and reader latency from 65.519 to 47.094 μs, compared with reading the KV cache that must scan every token.

Draft Model (1B) — generates K=5 tokens

Paris.Itis

↓ verify all at once↓

Target Model (70B) — one forward pass

✓ Par✓ is✓ .✗ It→The— is

accepted

rejected → corrected

discarded

3 accepted + 1 corrected = 4 tokens from 2 forward passes

↻ repeat until done

Where it earns its keep

Take the paper's averages on Qwen3.8-27B with block size 7 and greedy decoding. DSpark sustains 280.42 tokens/s at 4.17 tokens committed per verification; ReaDy sustains 302.78 tokens/s at 4.53. Dividing acceptance length by throughput gives the time of one draft-verify round (approximate, derived from the averages): about 14.88 ms for DSpark and 14.96 ms for ReaDy. The reader adds only about 0.07 ms, roughly 0.5%, per round, yet each round commits 8.5% more tokens, so throughput rises about 8% for about 0.5% more time per round. Plain autoregressive decoding runs at 95.86 tokens/s, which is how 302.78 becomes the headline 3.16×.

Benchmark	DSpark → ReaDy (tokens/s)	Throughput gain	All 7 drafts survive (DSpark → ReaDy)
GSM8K	362.25 → 393.60	+8.66%	45.98% → 58.17%
HumanEval	287.95 → 310.29	+7.76%	25.89% → 31.19%
MT-Bench	234.54 → 256.71	+9.45%	12.59% → 16.11%

Two controls in the paper keep this honest. Fine-tuning DSpark without state access already reaches 286.50 tokens/s, so part of the 7.97% is just extra training, and ReaDy beats that tuned control by 5.68%. And more reading is not better: adding a third source layer (layer 2) cuts the gain to 6.22%, rank 128 instead of 8 buys almost no extra acceptance (4.5404 vs 4.5285) at lower throughput, and injecting in all five draft layers leaves acceptance flat while throughput falls to 297.35 tokens/s. All numbers are for one B200, request concurrency one, temperature zero and one target model.

Goes deeper in: LLM Serving → Speculative Decoding → Draft Model Variants

Continue in trackSpeculative Decoding: the drafter variants ReaDy extends

Related explainers

JetSpec: parallel tree drafting — a different way to spend the draft budget: shape the draft as a tree
SwitchSD: copy-intent drafting — another drafter that reads the target's internal signals
Low-bit recurrent states — what the recurrent state ReaDy reads costs to store

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based