D3D is a standalone diffusion drafter for speculative decoding. It predicts a block of 16 candidate tokens in one pass **without target-model hidden states**, so it can prepare later blocks while the target verifies the current one.

Why does removing hidden states matter?

A drafter that consumes the target's hidden states must wait for verification to finish before it drafts again. Without that dependency, drafting can overlap verification, which is the asynchronous speculative decoding D3D demonstrates.

Is the output still the same as the target model's?

The paper describes the verification as rejection sampling with the proposal probabilities, the standard rule that preserves the target distribution, and calls the result lossless asynchronous speculative decoding.

What happens when the prepared block does not match?

That is a cache miss. The target waits while D3D drafts a new block from the verified prefix synchronously, so a miss costs about what a non-overlapped round costs.

LLM· 2026-11-06

D3D diffusion drafter — Asynchronous speculative decoding — What does it mean?

Jargon

Speculative decoding: A small, fast model proposes several tokens and the big model checks them all in one pass. Accepted tokens are kept, so the output matches what the big model alone would produce. Draft-verify, step by step
Target model: The large model whose output you actually want, here Qwen3-8B or Qwen3-32B. It does the verifying.
Drafter: The small model that guesses the next block of tokens. In D3D it is a diffusion model, so it fills in all positions of a block at once instead of one token at a time.
Hidden states: The intermediate vectors a model computes for each token while it runs. DFlash feeds the target's hidden states to its drafter; D3D does not.
Acceptance length (τ): The average number of output tokens added per verification round, counting the one token the target itself produces. D3D averages 7.88 with Qwen3-8B at block size 16.
Cache hit: The case where the verified prefix matches the start of a block D3D already prepared, so that block is reused and no new drafting pass is needed.
Layer pruning: Removing the layers that matter least from a model. D3D prunes its drafter by how little each layer changes acceptance length, so drafting finishes faster.

The news. On November 4, 2026, researchers from Fudan University and partner institutions posted D3D to arXiv: a standalone diffusion drafter that drafts the next blocks during verification. Across seven benchmarks it averages 5.00× and 5.71× decoding speedup over plain decoding with Qwen3-8B and Qwen3-32B targets. Read the paper →

Picture a kitchen where the head chef tastes each plate before it leaves, and the sous-chef plates the next course. In the usual setup the sous-chef waits for the head chef's notes on the plate just approved, and only then starts the next one. The pass is fast, but the two never work at the same time. That is how a target-conditioned drafter behaves: the next draft needs fresh hidden states from verification, so drafting and verifying run one after the other, even though the drafter fills a whole block in one forward pass. See how the draft-verify loop works for the base loop.

D3D removes the notes. Its drafter reads only the token prefix, never the target's hidden states, so it does not have to wait for verification to finish. Two training choices make that workable. Blockwise denoising teaches it to fill in a block of 16 tokens in one pass, and an auxiliary autoregressive loss on target-generated responses keeps the guesses coherent. Setting that loss weight to 1 raised acceptance length by 0.24 to 0.40 tokens over diffusion-only training in the paper's ablation. With these in place, D3D beat every baseline in acceptance length on all seven benchmarks.

Now the sous-chef can start early, but there is a catch: nobody knows yet where the head chef will stop approving. So the sous-chef pre-plates one tray for each plausible outcome, all in a single packed drafter pass. Each tray extends the current prefix by the tokens accepted before a rejection point, a different token at that point, and a fresh block after it. One more tray covers the case where every token passes. When verification ends, if the verified prefix matches a prepared tray, that tray is served immediately; if not, the target waits while D3D drafts synchronously, exactly as before. The paper takes this scheme from Speculative Speculative Decoding, which did it with autoregressive drafters, and shows it with a single-pass diffusion drafter. A related idea for looped models is covered in LoopSpec.

The size of the packed pass is a budget you choose: with block size B, rejection positions covered C, and F alternative tokens per position, the draft budget is B × [(C − 1) × F + 2] tokens. The next table shows why more coverage is not automatically better.

Setting (B = 16, AIME25)	Draft budget	Cache hit rate	Effect on speedup	Source
C = 12, F = 2	384 tokens	71.3%	Best, about 5.0×	Paper, Sec. 4.3
C = 16, F = 2	512 tokens (from the formula)	78.3%	Lower than C = 12	Paper, Sec. 4.3
C = 12, F = 5	912 tokens	81.8%	Lower than F = 2	Paper, Sec. 4.3

Read the table row by row: a higher hit rate bought a lower speedup, because every packed pass also computes trays that verification never picks. The paper therefore chooses the budget by measured speedup, not by hit rate.

Here is the arithmetic with (illustrative) timings, since the paper reports end-to-end speedups and not these latencies. Say one packed drafting pass takes 8 ms and one verification pass takes 20 ms, and a hit rate of 71.3%. Without overlap every round costs 8 + 20 = 28 ms. With overlap, a hit hides the draft entirely, so the round costs only the 20 ms of verification. A miss costs 20 + 8 = 28 ms, because the target waits for a fresh draft. The expected round is 0.713 × 20 + 0.287 × 28 ≈ 22.3 ms, so 28 ms falls to about 22.3 ms per round, about 1.26× faster before counting any change in acceptance length. The condition for a clean hide is that drafting finishes within the verification time, which is why a bigger budget can backfire.

That condition also explains the second D3D idea, acceptance-guided layer pruning: removing layers with the smallest effect on acceptance length, with recovery training between removals. On Qwen3-8B, dropping 5 of 36 drafter layers (13.9%) kept over 99% of mean acceptance length and raised mean speedup from 4.44× to 4.69× on three benchmarks. Two limits are worth keeping in mind. The gain shrinks as load grows: with the D3D-1.7B drafter and Qwen3-32B on the Transformers backend, MATH-500 speedup fell from 6.1× at one request to 2.7× at 16, while throughput still rose. And the study uses Qwen3 targets only, so the numbers are not a guarantee for other models or serving engines. For how drafters differ, see draft model variants.

Goes deeper in: LLM Serving → Speculative Decoding → The Draft-Verify Idea

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based