The news. On July 13, 2026, researchers published a study of interaction scaling that compares three ways to spend compute at answer time: longer reasoning, best-of-N sampling, and interaction with external instruments. On hard coding tasks, reasoning-only and best-of-N plateau while interaction keeps improving — one proposer-reviewer harness reaches a 100% pass rate with no run-to-run variance. For visual artifacts, an ungrounded VLM judge marks 14 of 15 visibly broken figures as perfect, while measurement-tool loops remove 40–74% of defects across four modalities. Read the paper →
Picture a darts player practicing alone. She can stand at the line and think harder about her grip, or she can hurl ten darts and keep the closest — but the throw that actually improves her aim is the one where she walks up, sees exactly where the dart bit the board, and adjusts. Now picture a blindfolded friend who shouts "perfect!" after every throw. That praise feels like feedback, yet it moves her aim nowhere, because it never saw the board. A feedback loop only makes the work better if the reviewer actually looks at where the dart landed.
Drop the metaphor and the mechanism is precise. The paper treats interaction as a third axis of test-time compute, separate from thinking longer and from sampling more. Interaction — propose, observe the real artifact, revise — is the only one of the three the paper reports still improving on hard coding tasks, because each pass is corrected by something that watched what actually happened. This is the difference between an agent that pauses to observe before deciding and one that reasons in a straight line off the single-shot failure and never checks its work.
But the loop has a catch, and it is the whole point. The feedback has to be grounded: the reviewer must observe the real result, not guess from a thumbnail. Hand the review to an ungrounded VLM — a model glancing at a rendered figure — and it will wave broken work through. The paper's sharpest result is that an ungrounded reviewer doesn't just fail to help; the study reports a loop built on it can make the work worse, because it keeps stamping "done" on things that aren't. Swap in a tool that actually measures the artifact — run the layout, diff the numbers — and the loop repairs the defects instead. It is the eval question of pass/fail versus a real score, pushed into the training-free feedback loop.
Not every way of spending compute uses feedback from the real world, and that is exactly what separates them:
| Way to spend compute | What it does with the budget | Feedback from the real artifact? | On hard coding tasks |
|---|---|---|---|
| Longer reasoning | think more before answering once | No — never leaves the model's head | plateaus |
| Best-of-N sampling | draw many answers, keep the best | No — guesses don't learn from each other | plateaus |
| Interaction (grounded) | propose → measure → revise, in a loop | Yes — an instrument observes the real result | keeps improving |
Here is the gap made concrete. Take the visual-artifact case. The study points an ungrounded VLM at 15 visibly broken figures and it calls 14 of them perfect — a derived catch rate of roughly 1 in 15 (~7%), just arithmetic on those reported counts. As a feedback signal that is close to useless: on fourteen of the fifteen it tells the agent "you're done," so revision never fires and the loop can't improve what it can't see. Swap the reviewer for a measurement tool that inspects the actual layout, and across four modalities a propose-and-revise loop removes 40–74% of the defects it's pointed at. What separates the two reviewers is whether one observed the real artifact or guessed from a thumbnail.
The uncomfortable takeaway for anyone wiring up an agent is that the reviewer, not the model, can be the bottleneck. Grounding the loop doesn't make the model smarter; it makes the model's mistakes visible enough to fix — which is why an online eval you can trust can be more trustworthy than a green dashboard built on the agent's own say-so. Interaction scales because the darts player finally takes off the blindfold.
Goes deeper in: AI Agents → Planning & Reflection → When to Pause and Observe
Related explainers
- LLM-as-a-Verifier — Verification as a scaling axis — the complement on the eval side: read the verifier's whole score spread instead of one pass/fail label
- Vera — Evidence-grounded verification — the same "judge the real artifact, not the self-report" idea, applied to agent safety in a sandbox
- The Verification Horizon — Co-evolving verifiers — why a fixed reviewer saturates as the agent improves, and the loop has to co-evolve
- QVal — Q-aligned dense supervision — a training-free look at how dense a feedback signal actually needs to be