What is difficulty-aware Think vs NoThink routing?

It is a hybrid reasoning model deciding, per problem, whether to emit an explicit reasoning trace (Think) or answer directly (NoThink), and how long that trace should run. When2Think trains the decision into the weights using two precomputed numbers per training problem: how often a reference copy of the model got it right, and how many tokens it usually took.

Long reasoning traces are where a thinking model spends its inference tokens, and a fixed budget is wrong in two directions at once — too long on easy problems, too short on hard ones. Making length an allocation decision rather than a global setting is what let accuracy and token usage move together in the paper's AIME24 run, instead of trading off.

How is it different from a uniform length penalty?

A uniform penalty shortens every trace by the same rule, so its savings come partly out of problems that needed the length. In the paper's AIME24 numbers the compression baseline LC-R1 drops 10.0 points of Pass@3 against the base model. When2Think measures each problem's expected length first and penalises overrun relative to that, so the pressure falls on the easy problems.

How does it relate to routing a query to a cheaper model?

Both are per-request allocation decisions, but they pull different levers. A cost-aware cascade picks which model answers; difficulty-aware Think routing keeps one model and picks how much it deliberates. They compose — you can route to a cheap model that also knows when to skip its own reasoning trace.

LLM· 2026-09-20

When2Think — Difficulty-aware Think vs NoThink routing — What does it mean?

TL;DR

What is it: When2Think (arXiv 2609.19671, Sungkyunkwan University and Microsoft, 17 September 2026) is a post-training recipe that teaches a reasoning model difficulty-aware Think vs NoThink routing — deciding, per problem, whether to write out a reasoning trace at all and how long it should run.
Why it’s needed: A thinking model pays for its answer in generated reasoning tokens, and the waste runs in both directions at once: these models overthink easy problems and underthink hard ones, so one fixed budget misallocates at both ends.
vs previous: The standing fixes pay an efficiency tax. Length-compression methods (LC-R1) and routing-only hybrids (AdaptThink) buy their token savings with accuracy lost on the hard instances; When2Think instead prices every trace against that problem's own reference length, so it can cut the easy ones and still spend on the hard ones.

Jargon

LRM: Large Reasoning Model. A model post-trained to write out an explicit reasoning trace before its answer, which can raise accuracy on problems that benefit from the extra deliberation — and burn tokens on problems that do not.
Think / NoThink: The two modes of a hybrid reasoning model. Think emits a reasoning trace before the answer; NoThink answers directly. When2Think learns which mode each problem gets.
IDAC: Instance-level Difficulty-Aware Control. When2Think's reward-shaping term. It multiplies the efficiency bonus of a Think trace by a factor that shrinks exponentially with the trace's length, scaled by how easy the reference runs found this problem and divided by how long they took — so the penalty bites hardest on the problems that needed the least.
Efficiency tax: The paper's name for the accuracy you give up on hard problems in exchange for token savings on easy ones. A method pays it when its cheaper average response comes out of the problems that actually needed the tokens.
Pass@3: Sample three answers per problem and count the problem solved if any one of them is correct. It rewards a model that can find the answer at least occasionally, which matters on competition mathematics.
RLVR: Reinforcement Learning with Verifiable Rewards. The reward is a checker's verdict on the final answer rather than a learned preference model. Full explainer
BWS: Batch-wise standardized advantage. When2Think standardizes each trajectory's reward across the other problems in the batch instead of within one problem's own samples, which keeps the difficulty differences between problems visible to the optimizer. How GRPO standardizes instead

The news. On 17 September 2026, researchers at Sungkyunkwan University and Microsoft posted When2Think, a post-training framework that teaches a hybrid reasoning model to allocate its own reasoning budget problem by problem. Training a 1.5B R1-distilled model on competition mathematics, they report Pass@3 on AIME24 rising from 46.0% to 56.0% while average tokens per response fall 27.9%, and 40.0% Pass@3 on AIME25. Read the paper →

Walk into the clinic with a splinter and you do not want the full consultation. Walk in with chest pain and you very much do. The triage desk exists because the right amount of the doctor's time is not a property of the clinic, it is a property of the patient in front of it — and a clinic that gives everyone the same fifteen minutes is simultaneously wasting them on the splinter and rationing them on the chest pain.

That is the state a large reasoning model is in today. Its trace length does move with difficulty — the untuned baseline here, R1-Distill-Qwen-1.5B (a 1.5-billion-parameter model distilled from DeepSeek-R1), averages about 1,199 tokens on the easiest tier of MATH-500 and about 6,830 on the hardest — but moving is not the same as matching. The paper's case is that the allocation is wrong at both ends: 1,199 tokens is still a long consultation for a question that needs almost none, and on the hard tier the methods that trim the total start cutting into reasoning the answer depended on. When2Think moves the triage decision off the harness and into the model's own weights: the same network that answers the question also decides, before it starts, whether this one gets the fast lane or the consultation room.

The training signal is where the metaphor gets specific. Before each epoch, a reference copy of the model runs the whole training set offline, sampling K answers per problem, and the run is reduced to two numbers per problem: α, a scaled measure of how often the reference got it right, and τ, how many tokens it typically took. Those two numbers are the triage desk's file on that patient — a difficulty score and an expected length, measured rather than guessed.

IDAC then prices each new trace against that problem's own τ rather than against a global budget, which is what lets the same 8,000-token trace be extravagant on an easy problem and reasonable on a hard one. Concretely, a Think trace's efficiency bonus is scaled by a factor that decays with the trace's length times α over τ, so exceeding the reference length hurts most where the reference says the problem was easy; a NoThink answer takes the factor undiminished. The bonus is gated on being correct, so the model cannot collect it by answering short and wrong. Two supporting pieces make that signal trainable without a learned critic: BWS standardizes each trajectory's reward across the other problems in the batch, which preserves the difficulty ordering the reward just encoded, and an importance-sampling step forces the first mode token to be drawn uniformly during rollouts (the sampled attempts the model learns from) so the model actually explores both lanes instead of collapsing into the one it already prefers. Correctness comes from a symbolic verifier and the reference statistics are computed offline, so nothing in the inner loop needs a reward model or a second model online.

Reference runsK samples per problem, offline, once per epoch

α and τreference accuracy and reference length

IDAC rewarddecay a Think trace by its length times α over τ

BWS advantagestandardize across the batch, not within a problem

Policy updatecritic-free: no learned reward model, no second model online

The comparison that carries the argument is AIME24 — the 2024 American Invitational Mathematics Examination problems, one of the paper's hardest competition benchmarks — and it is worth naming what is held fixed across the rows below: the same R1-Distill-Qwen-1.5B backbone, the same problem set, matched decoding settings, and Pass@3 averaged over five independent runs. Only the efficiency method changes.

Method on AIME24	Pass@3	Tokens / response	vs base	Source
R1-Distill-Qwen-1.5B (base)	46.0%	14,195	—	Table 1
LC-R1 (length compression)	36.0%	7,972	−10.0 pts, −6,223 tokens	Table 1
AdaptThink-δ0.05 (mode routing only)	44.7%	5,806	−1.3 pts, −8,389 tokens	Table 1
Thinkless-RL (mode routing only)	46.7%	11,023	+0.7 pts, −3,172 tokens	Table 1
When2Think	56.0%	10,236	+10.0 pts, −3,959 tokens	Table 1

Deciding whether to think is not enough on its own: AdaptThink makes that decision and still finishes at 44.7%, below the untouched base model it was supposed to improve. It cuts the average response from 14,195 tokens to 5,806, and the saving comes out of the problems that needed the tokens — that is the efficiency tax, paid in full. The difference in When2Think is that the reference length gives it a second dial: not only whether to open the consultation room, but how long the consultation runs, tuned to what that problem has historically cost. Across MATH-500's five difficulty tiers the resulting Think ratio climbs monotonically from roughly 0.2 at Level 1 to over 0.7 at Level 5, which is the behaviour a triage desk is supposed to produce.

Price it out, and hold three things fixed: the AIME24 problem set, the 1.5B backbone, and Pass@3 scoring, which samples three answers per problem and counts it solved if any one is right. Normalize to 100 AIME24-style problems, three complete samples each, with a ground-truth checker deciding what counts as solved. The base model writes 14,195 tokens per response, so three samples each is 3 × 14,195 × 100 ≈ 4.26M tokens, and 46 problems come back solved — about 92,600 tokens per solved problem. LC-R1 looks like the bargain at 7,972 tokens a response, which drops the bill to ≈ 2.39M, but it solves only 36, so each answer you keep cost about 66,400 tokens. When2Think writes 10,236 tokens a response — more than LC-R1 — for ≈ 3.07M tokens and 56 solved: about 54,800 tokens per solved problem, roughly 41% below the base model and 17% below the cheaper-looking compression baseline. The per-solved figures are ratios derived from the paper's Table 1 rather than measured deployment bills reported in it, and the token counts are means over five runs. This is the same reason cost per successful outcome reorders models that price-per-token ranks the other way, and the reason an agent's budget is easier to reason about as a cost profile than as a token count.

Everything in that table is one 1.5B math reasoner on competition mathematics, so treat the mechanism as the transferable part and the numbers as strictly local. Three conditions travel with it. The reference statistics are not free: K sampled trajectories over the whole training set, re-run before every epoch, which is real compute spent to avoid spending compute later. The recipe needs a verifier that can call an answer right or wrong, and mathematics has one where an open-ended agent task usually does not. And the difficulty score is the reference model's own accuracy, so "hard" here means hard for this model — a definition tied to whichever reference policy produced the statistics, which are precomputed fresh before each epoch. What plausibly generalizes is the reframing: reasoning length is not a compression problem to be minimized but an allocation decision to be made per request, and the decision needs a per-request reference to be made against. The same question shows up one step later in the loop as when to stop, and one layer down as how many tokens a single generation actually costs.

Goes deeper in: AI Agents → Planning & Reflection → When to Spend More Tokens

Related explainers

Microsoft MAI-Code-1-Flash — Adaptive solution-length control — the same decision shipped as a product capability, where the reported result is the token saving rather than the training recipe behind it.
Cluster-Route-Escalate — Cost-aware LLM cascade — the other place this decision can live: route the query to a different model instead of the model to a different depth.
Cost per successful outcome — why the cheaper response is often the more expensive answer.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based