LLM·

HDM hybrid decision model — Value-of-thinking routing — What does it mean?

The news. On November 12, 2026, a group of researchers posted "The Value of Thinking: Learning When to Reason in Hybrid Decision Models" on arXiv. It wraps one frozen Qwen backbone (Qwen3.5-4B or Qwen3.8-27B) in a fast mode that writes no tokens and a slow mode that reasons, with a learned router choosing between them once per question. On JevBench the 4B version scores 0.857 at a 0.179-second median, against 0.851 at 0.652 seconds for Jev, a commercial decision model that also answers in one pass without generating, and on the MiniWoB agent benchmark the 27B version cuts thinking tokens by 80.1% for 0.05 lower reward. Read the paper →

Picture the appraiser at the counter. A customer hands over a ring, and she has two ways to price it: quote on sight, with one look and no fee, or send it to the lab for a test that takes a day and costs money. The useful question is not "how sure am I?" but "would the lab change my answer?" A brass ring she is sure about never needs the lab. A strange alloy that the lab cannot identify either does not need it — she would pay the fee and still quote wrong. Only the ring where the test turns a wrong quote into a right one is worth the fee.

HDM gives a language model the same two options. In the fast mode, the prompt lists the candidate answers with short codes and ends just before the answer, so a single forward pass leaves the answer in the next-token logits over those codes, with zero tokens generated. In the slow mode, the same model writes its normal chain of thought, one token at a time, up to 4,096 tokens. Both modes are rendered from one prompt, so an escalated question continues from the context the fast pass already encoded instead of paying for a second prefill.

The choice is made by the self-routing block, which reads what the model noticed during the glance: its hidden state at the answer position, plus eight statistics of the fast answer's probabilities. It predicts the four joint outcomes — both modes right, only the fast one, only the slow one, neither — and from them computes the value of thinking, v = log(P(slow right) / P(fast right)). A question is escalated only when v is greater than the price λ set on that request, which means only when thinking is predicted to raise the chance of a correct answer by a factor of at least eλ. The paper argues that confidence cannot stand in for v, because the two come apart in both directions: thinking sometimes spoils an answer the fast mode had right, and it adds nothing where both modes are right. Its ablation backs the use of the hidden state: the best hidden-state configuration ranks the questions where thinking helps with a routing AUROC (a ranking score where 0.5 is random and 1.0 is perfect) of 0.778, against 0.671 for the same block fed only the fast answer's statistics.

Because λ is an input and not a trained constant, one trained router covers the whole cost-accuracy curve. Training draws a different λ for each example between −0.2 and 0.5, and at serving time a latency-sensitive request can carry a high price while an offline batch job carries a low one. This turns the reasoning-budget decision into an explicit, priced choice — a routing step between two modes of one model rather than between two models.

In an agent task, no single step has a right answer to train against; only the episode's final reward exists. For that case the paper trains the router with CDPO, a PPO-style recipe that scores each episode by its final reward multiplied by e−λ × number of escalations, so every escalation is charged against the agent's cost profile. On ALFWorld at λ = 0, the router reached a 0.560 success rate, between always-fast at 0.463 and always-think at 0.642: 54% of the available gain for 48% of the thinking tokens.

Routing ruleQwen3.8-27BQwen3.5-4BWhat it routes onSource
Always think5911,709Every question goes to the slow modePaper, Table 6
Random escalation213991A coin with probability p, sweptPaper, Table 6
Confidence threshold138945How sure the fast answer isPaper, Table 6
ThinkSwitcher (reproduced)3521,709Predicted success rate of each modePaper, Table 6
RouteLLM (reproduced, two-mode version)112822A binary classifier on the same inputs HDM readsPaper, Table 6
HDM103792Predicted value of thinking, v > λPaper, Table 6

The table holds the target fixed — 90% of always-thinking's accuracy on JevBench's 231 public items — and reads the thinking tokens each rule spends per question to reach it. Predicting the gain beats predicting correctness: the confidence rule needs 138 tokens a question on the 27B model, HDM 103. The margin over RouteLLM, refitted on the same hidden state and statistics, is small (103 against 112), and the paper notes that the ordering depends on where the target is set. ThinkSwitcher is the warning case: it cost more than escalating at random on the 27B model, and as much as always thinking on the 4B.

Here is the trade-off worked through on the paper's main test: Qwen3.5-4B on three MMLU-Pro subject categories held out from training (346 questions), with thinking capped at 4,096 tokens. Always fast scores 54.3% with zero tokens; always thinking scores 85.3% at 2,281 tokens and 12.3 seconds per question. At λ = 0 — escalate whenever thinking is predicted to help at all — the router sends 66% of questions to the slow mode and scores 84.4% at 1,845 tokens and 9.9 seconds: 0.9 points of accuracy given up for 19% fewer tokens and 19% less wall-clock time. Raise the price to λ = 0.5 and escalation falls to 39%, for 77.2% accuracy at 1,166 tokens: 49% fewer tokens for 8.1 points. To see the price on one question (illustrative): if the router predicts the fast answer is right with probability 0.50 and the slow answer with 0.60, then v = ln(0.60 / 0.50) ≈ 0.18. That clears λ = 0.1, so the question is escalated; at λ = 0.5, thinking would have to make a right answer at least e0.5 ≈ 1.65 times as likely, so the question stays fast. One caveat: the slow mode hit its 4,096-token cap on 36.1% of the 4B model's questions, so the paper treats its token counts and accuracy as lower bounds.

Keep the size of the claim in view. The fast mode is built for decision-style requests — choosing among listed candidates or predicting a number — and the router for single-question tasks was trained on MMLU-Pro categories, then transferred unchanged to the other benchmarks. The price is also charged per escalation using the average trace length, so, as the paper says, it escalates too readily where a trace would run long and too rarely where it would be brief. What lasts is the criterion: spend reasoning where it is predicted to change the outcome, not where the question looks hard or the quick answer looks unsure.

Goes deeper in: AI Agents → Planning & Reflection → When to Spend More Tokens

Related explainers

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based