What is controller-worker separation?

It is an evaluation design that scores the model supervising a coding agent while holding the coding agent itself fixed. The controller reads a structured summary after each round and decides the next move; the worker carries it out. Because only the controller changes between runs, the largest confound is removed and the score reflects guidance far more than coding ability.

Why does separating the controller from the worker matter?

A single end-to-end pass rate mixes two independent skills. A loop can fail by trusting a stale progress note, skipping a verification, misspending its budget, or stopping too early — none of which are coding errors. Without the split you keep tuning the coding agent for failures the loop caused.

How does LoopArena actually measure a controller?

In three settings of increasing execution cost. The cheapest scores the next-step Loop Contract choice without running the worker; the middle one runs repeated control over a slice of a task; the most expensive runs the complete paired task from its original state. The best observed Strict Success Rate on full tasks was 24.69%.

How is this different from the supervisor and worker pattern?

It is the same topology used as a measuring instrument rather than an architecture. Supervisor and worker tells you how to build a multi-agent system; controller-worker separation tells you how to attribute its failures once it is running.

Agent· 2026-09-07

LoopArena benchmarks the model that supervises a coding agent — Controller-worker separation — What does it mean?

Jargon

Controller: The model under evaluation. After each coding round it reads a summary of the run and tells the worker what to do or verify next, or that the work is finished.
Worker: The coding agent that does the actual work. It is fixed across every controller under test, which is what makes the comparison attributable.
Loop Engineering: Designing the loop around a coding agent — what it monitors, what it assigns, what it checks — instead of hand-writing each prompt. The paper names this as the practice it is trying to measure.
Loop Contract: The set of next moves a controller can choose from at each round. The cheapest evaluation setting scores that choice directly, without running the worker at all.
Strict Success Rate: The pass metric on full paired tasks. The best controller in the paper reached 24.69%.
Paired evaluation: Running two conditions over the same task set from the same starting state, so their difference is attributable rather than coincidental. Slice evaluation leans on the same idea.

The news. On August 28, 2026, the LoopArena team published a benchmark that scores a model on how well it guides a separate, fixed coding agent through a long-running task. The model under evaluation is the Controller; the coding agent it directs is the Worker. The benchmark runs three settings that differ in how much they actually execute, and reports that the best observed Strict Success Rate on full tasks is 24.69%. Read the paper →

Picture the rally car halfway through a stage. The co-driver has the pace notes, calls the corner, and the driver executes it. When the car goes off the road, the team is left with one stage time and two suspects: did the co-driver call the corner wrong, or did the driver miss the apex? A single end-to-end number cannot separate them, and that is exactly the problem controller-worker separation is built to fix.

The fix is to bolt the driver in place. You freeze the worker — the same coding agent, the same tools, the same starting state — and swap only the controller, the model that reads the run summary after each coding round and decides what happens next: do more work, verify something, or stop. What is held identical across runs is the worker and everything handed to it, not the trajectory — each controller steers that somewhere different, which is exactly the point. Fixing the worker removes the single largest confound, so a difference in outcome now points at the controller rather than at the coding agent.

That decision is where loops actually break. The paper names four ways a loop goes wrong even with a capable coding agent underneath: it trusts a stale progress note, skips a verification it needed, spends its budget in the wrong direction, or stops before the task is safe to submit. None of those are coding mistakes. They are supervision mistakes, and an end-to-end score quietly charges every one of them to the coding agent.

If that split sounds familiar, it is the supervisor and worker topology you already know, turned into a measuring instrument. The same shape shows up in workflow design as orchestrator-workers. The point here is not how to build one — it is that you cannot debug one until your metric stops averaging the two roles together, which is the same discipline as doing error analysis before you tune anything.

Worker runsfixed coding agent

Run summarystructured, per round

Controller decidesact, verify, or stop

Next instructionback to the worker

What you run	What stays fixed	What the score lets you conclude
A conventional end-to-end agent benchmark	nothing — the model and its harness move together	That this whole configuration passed or failed. Not which half caused it.
The paired full task, controller swapped	the worker, its tools, and the starting state	How much of the outcome the controller owns. Best strict success: 24.69% (source).
The next-step choice only	the worker is not executed at all	Whether the controller picks the right next move, before any code runs.

Hold three things fixed and the arithmetic gets honest. Take one task set, one worker, and one starting state, then run two controllers across it. Controller A finishes 24.69% of the paired full tasks strictly — the best figure the paper reports. Controller B, on the same set, finishes 18% (illustrative). The gap is 6.69 points.

Now ask what could account for those 6.69 points. Not the coding agent: it is the same agent in both columns. Not the task set, the tool list, or the starting state: all held. Not sampling luck across different tasks: the comparison is paired, the same tasks both times. What is left is loop control, so the 6.69-point gap is attributable to the controller rather than the coder — subject to ordinary run-to-run variance, which is exactly why the comparison is paired. Run those two models through a conventional end-to-end benchmark instead and the same subtraction carries a coding-ability term you can neither measure nor remove.

Goes deeper in: Agent Engineering → Agent Teams → Supervisor / Worker

Related explainers

Slice evaluation as a rank-preserving proxy — once the controller is isolated, this is how you rank controllers without paying for a full run each time

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based