Gate risky coding-agent actions with draft-model uncertainty — Speculative Uncertainty — What does it mean?

The news. On September 4, 2026, a paper titled How to Speculate about Uncertainty in Agentic Coding? A Draft-Model Gate Method introduced Speculative Uncertainty. A small open-weight draft model scores a black-box coding agent's already-generated trajectory in a single forward pass; the resulting failure score feeds a pre-execution veto gate. Tested on Qwen3-Coder-480B and Claude 3.5 Sonnet, it cut execution error rate by 6–8 percentage points and token cost by 14–19% in deployment, and the authors report it transferred to benchmarks it had not been tuned on, without retraining. Read the paper →

Picture the letter again. It is written, it is sealed, and it is one hand-movement away from the postbox. Once it drops through the slot, no amount of regret gets it back — which is exactly the shape of an agent running rm, opening a pull request, or pushing a migration. The expensive part of an agent's mistake is not deciding badly; it is that deciding and doing are the same motion, so the cost is already paid by the time anyone notices.

So you want a reader at the postbox. Not a second author — a second author is slow and costs as much as the first. You want somebody cheap who reads the finished letter once, straight through, and reports only how much of it surprised them. That is the whole idea: the surprise of a cheap reader is a usable predictor of whether the expensive writer's work will fail. The draft model never proposes a better letter. It just flinches, or doesn't.

The letter is not uniform, and neither is a trajectory. Most of it is the agent explaining itself — reasoning spans, discursive, where unusual phrasing means very little. A small part of it is the line that acts: the command, the diff, the API call. SU separates these reasoning and action spans and reads surprise separately in each, so the two contribute distinct features instead of one blended number; the paper does not say how much each span ends up mattering. Those features are then calibrated against an objective the authors could check mechanically, which is what turns raw surprise into the failure-likelihood score a policy consumes.

That diagram is the arrangement you already know from the draft-verify idea: the small model runs first, guessing tokens that do not exist yet, and the big model checks the guesses in one pass. Every draft-model variant in that module is tuned for the same goal — more accepted tokens per forward pass, meaning speed.

Speculative Uncertainty runs the same two models in the opposite order, for a completely different prize. The big model goes first and writes the whole trajectory. Then the small model reads what already exists, in one forward pass, and the output is not tokens at all — it is a number saying how badly the two models disagree about what should have been written. Speed was never the point. The point is that this disagreement turns out to predict failure, and that measuring it needs nothing from inside the big model, which is the difference between a method that works on your own weights and one that works on an API you rent.

Where the savings actually come from

The surprising claim is not that the gate catches mistakes — it is that gating lowers the token bill while adding a model to the pipeline. Hold one quantity fixed and it comes apart cleanly. Take 1,000 agent runs, and say each one costs 50,000 tokens and 30% of them end in an execution error (both baselines illustrative — the paper reports changes, not absolute levels).

Without the gate: 50,000 × 1,000 = 50M tokens, and 300 failed runs. Now add SU. It charges one draft-model forward pass per trajectory — a read, not a generation, on a model far smaller than the agent, so it barely registers against 50,000 tokens of writing. Against that, the paper's measured deployment effect is a 14–19% cut in token cost, landing the bill at 40.5M–43M tokens — 7–9.5M tokens saved, after paying for the gate. The error rate falls 6–8 points at the same time, from 30% to 22–24%, or 60–80 fewer failed runs.

So the bill falls even though you added a model to the pipeline. The paper does not break that 14-19% down, but its own framing of the problem points at where the room is: bad actions, it says, are recognised only after costly execution and retry. A vetoed action never executes, so it never produces a broken state, a stack trace, and a retry that re-reads the whole context to try again — and on that reading, skipping the retry loop pays for a lot of cheap reads. This is the same accounting as deciding when to pause and observe — the cheapest step is often the one that stops you doing an expensive one.

Two things are worth keeping in proportion. First, the score is a risk signal, not a verdict: it is a layer, and it belongs in a stack with the input and output filters covered in defense-in-depth, not in place of them. A gate with a threshold has false positives, and a vetoed-but-correct action costs you something too. Second, the veto gate is only the demonstration. The paper is careful that the output is a score any downstream policy can consume — route the risky ones to a stronger model, send them to a human, or spend more compute on them at inference time. Choosing the policy is a design decision about how reversible your actions are, which is why idempotency and this score are best thought about together: the less safely an action can be re-run, the more a pre-execution number is worth.

Goes deeper in: LLM Serving → Speculative Decoding → The Draft-Verify Idea

Related explainers

• Agentic abstention: when to stop — the other half of the question: once you have a risk signal, what does the agent do with it?
• PolicyGuide: workflow graph vs action veto — a different way to stop a bad action, by constraining the graph instead of scoring the trajectory.
• LLM as verifier: the logit-score scaling axis — what you can do when you do have logits, which is the access SU deliberately gives up.