The news. On July 9, 2026, the authors of DominoTree posted their paper (arXiv:2607.08642). It targets speculative decoding for Domino-style block-diffusion drafters — drafters that guess a whole block of tokens in one parallel pass. Instead of scoring each position on its own, DominoTree scores each root-to-node path through the draft tree with Domino's conditional, non-factorized GRU correction, and builds the tree with a GPU-native CUDA-graph procedure the paper says is bit-identical to its reference implementation. It reports up to 6.6× speedup over standard autoregressive decoding on Qwen3-4B and up to 10.7 accepted tokens per verification round. Read the paper →
Picture a group of friends trying to guess a sentence, all blurting a word for every slot at the same time. It is quick — every position gets a guess in one breath — but nobody heard the words before theirs, so if you just take the loudest guess in each slot you often get a string that does not read: "the… quickly… mountain… is." That is exactly the weakness of a block-diffusion drafter: it fills a whole block of tokens in parallel, scoring each position as an independent marginal, so the guesses are fast but do not always follow one another. The target model, checking the block, throws away everything after the first word that does not fit — the same unforgiving verification that governs all speculative decoding.
DominoTree keeps the parallel speed but adds one reader. Rather than trust each slot's loudest guess, it lays the candidates out as a tree — several options per position — and scores each root-to-node path by how well each token follows the one before it, using Domino's conditional GRU correction instead of the drafter's independent marginals. To stay cheap it only corrects the top-M candidates at each node rather than every branch. The target model then walks the re-scored tree and keeps the longest path that matches what it would have generated — and because the paths were chosen for how their tokens connect, that accepted run is longer.
The second half of the contribution is where the friends' game meets the GPU. Re-scoring a tree every decode step could drown the speedup in CPU overhead if the host has to re-issue the work each round. DominoTree instead builds the tree with a GPU-native CUDA-graph procedure — the whole build captured as a replayable graph the paper reports is bit-identical to its reference Python. It is the same reason CUDA graphs help ordinary decode: much of the per-step launch cost is paid at capture rather than re-issued from the host on every round.
Where it earns its keep
Standard autoregressive decoding advances the sequence by one token per expensive target-model pass. Speculative decoding changes the unit of work: one target pass now verifies a whole drafted block and keeps every token that matches, so the lever becomes acceptance length — how many drafted tokens survive each round. That is exactly what conditional path scoring lifts: a naive parallel block, scored position-by-position, tends to stop matching early and yields a short accepted run, while scoring whole paths by how each token follows the last keeps a longer run coherent. On Qwen3-4B the paper reports acceptance length rising to up to 10.7 tokens per round — so a single target pass advances the sequence far more than the usual one token — and, after the drafting and verification it pays for, end-to-end decoding lands at up to 6.6× faster than standard generation (arXiv:2607.08642). The block is drafted at the same parallel speed; the only thing that changed is that the tree was scored by how its tokens connect, not one token at a time.
| Draft strategy | How positions are scored | What the target tends to accept |
|---|---|---|
| Linear draft (vanilla spec-decode) | one straight line, token-by-token | the prefix up to the first miss |
| Static draft trees (independent heads) | several candidates per position, scored largely on their own | more coverage, but branches that need not cohere |
| Base Domino block-diffusion decoder | a fast parallel block, with Domino's own conditional correction on the draft | a fast block, a shorter accepted run |
| DominoTree conditional draft-tree scoring | root-to-node paths, each token conditioned on the ones before (top-M) | longer runs — up to ~10.7 accepted tokens/round, ~6.6× on Qwen3-4B (arXiv:2607.08642, setup-dependent) |
Because the change lives in how the draft tree is scored rather than in the target model's weights, it stacks on top of the rest of the serving toolkit, and the reported wins are relative to strong baselines — the paper claims roughly 9–10% more throughput than the released Domino decoder on Qwen3-4B and up to +24% over a competing draft-tree method (DDTree) on Qwen3-8B at greedy decoding (arXiv:2607.08642, setup-dependent). The headline is not a new drafter; it is a better-scored guess — one that stops a fast parallel block from being wasted on tokens that do not follow each other.
Goes deeper in: LLM Serving → Speculative Decoding → Draft Model Variants
Continue in trackSpeculative Decoding — the draft-verify loop DominoTree re-scoresRelated explainers
DominoTree sits in a cluster of spec-decode work that each attacks a different part of the draft-verify loop — the drafter, the block's neighbor-awareness, or the block size.
- JetSpec — Parallel tree drafting — also drafts a tree, but its whole point is a causal draft head; DominoTree instead re-scores a block-diffusion drafter's tree with a conditional correction
- DeLS-Spec — Long-short logit fusion — the sibling fix: makes a block draft read its own neighbors, the same independence problem from a different angle
- BlockPilot — Instance-adaptive draft block sizing — tunes how big the diffusion draft block is; DominoTree tunes how the block's tree is scored