Parallel Agent Teams — Many small teams vs one big team — What does it mean?

The news. On November 6, 2026, researchers from Duke University, Modal Labs, Princeton University and New York University posted "Test-Time Compute Scales Better with Parallel Agent Teams" on arXiv. They ran Qwen3.8-27B agents in the Codex harness, 12 hours per agent, on 12 long-horizon tasks from six benchmarks, from GPU kernels and formal proofs to exploit development. With a budget of 100 agents, ten teams of ten beat 100 independent agents on 10 of 12 tasks, and beat one team of 100 on all four tasks where such a team was run, by 8% to 30%. Read the paper →

Picture a contest with 100 cooks and one judge who tastes everything and keeps only the best dish. You can use the cooks in three ways. Send all 100 to cook alone, and the judge gets 100 dishes, but each one comes from a single person working in isolation, so the typical dish is weak. Put all 100 in one giant kitchen, and they pool what they learn, but the judge gets one dish, and large teams can converge on the same idea. Or build ten kitchens of ten: each kitchen shares a notice board, so its dish is much better than a solo cook's, and the judge still has ten different dishes to choose from. The judge's best pick depends on two things, how good the typical dish is and how much the dishes differ, and only the middle option gets both.

In agent terms, the judge is the verifier and a dish is a sample: one agent, or one team, scored by its best verified submission. This is best-of-N sampling with one change: the unit you sample is a team, not an agent. The paper's teams are flat. Each agent has its own sandbox, and all ten share a forum where anyone can post and read notes at any time.

The paper explains the result with a short statistical model. Assume one sample's score is roughly normal, with mean μ and standard deviation σ, and samples are independent. Then the expected best of n samples is μ plus σ times the expected maximum of n standard-normal draws. That second factor grows very slowly: from standard normal tables it is about 1.54 for n = 10 and about 2.51 for n = 100. So the mean counts in full at every budget, while each extra sample adds less than the one before. A team of k agents costs k agents, so a budget of B agents buys only B/k samples.

The paper's own toy shows the shape. Five runs score 2, 4, 5, 7 and 9. The expected best is 5.4 for one run (the mean), 7.1 for two runs and 9 for all five. The first extra sample adds 1.7 points; the last three together add 1.9.

Independent agents lose on the mean, and one big team loses on the spread. Independent agents saturate early: at 20 agents, every task had already reached 60% to 94% of the gain that about 100 agents bring. One big team is a single sample, so its expected score is just its own mean, and a bigger team does not reliably have a higher mean. From 64 to 100 agents, the one-team score dropped on all four tasks where it was run, and on two of them one team of 100 scored below the best of 100 independent agents.

The forum logs show why large crowds converge. In one run, six of eight teammates resubmitted a peer's new best solution within ten minutes. Copying a good idea helps a small team, but in a large team everyone ends up on the same idea, and the diversity that exploration needs is gone. This is the when teams beat a single agent question with a new answer: a team helps, but only up to a size.

Here is the model with the measured numbers for one task, Graph Node Classification. One agent: mean 52.1, standard deviation 11.8. One team of ten: mean 79.0, standard deviation 8.5. At a budget of 100 agents, independent agents give 100 samples, so the expected best is 52.1 + 11.8 × 2.51 ≈ 81.7. Teams of ten give only 10 samples, so the expected best is 79.0 + 8.5 × 1.54 ≈ 92.1. The teams lose on the spread term (13.1 points of variance gain instead of 29.6), but they start 26.9 points higher on the mean, which is far more. The model predicts a +12.7% gain for teams on this task, and the measured gain was +17.0%.

Because teams draw fewer samples, they are still on the steep early part of the best-of-n curve, so each added agent buys more. From 10 to 20 agents, each extra agent added 1.27% to the score of parallel teams and 0.64% to the score of parallel agents, so the gap between the two methods widens as the budget grows. This is the same budgeting question as when to spend more tokens, one level up: the test is always whether the higher mean pays for the samples you give up.

The same test rules out two other ways to spend the budget. A 12-hour agent costs as much as two 6-hour agents, but on Graph Node Classification the longer session raised the mean only from 52.0 to 52.7, while a team of two raised it by 5.4 points. And the team must stay flat. When the authors replaced the forum with an orchestrator that assigns work and collects reports, the orchestrated teams scored below both flat teams and independent agents at every budget on three tasks. On Borden Source Inversion, the best of ten orchestrated teams scored 68.4, below the average flat team at 76.5. The orchestrator is a bottleneck: workers wait for it and share findings only through it, which the authors identify as a bottleneck that lowers the orchestrated teams' mean, a coordination cost in practice.

The rule has limits. It needs a verifier that can score every attempt, and it assumes scores are close to normal. Teams lost on Kimi-Linear Decode (−7.9% measured) and, by a small margin, on Active Learning, and the model predicted the wrong sign on 3 of 12 tasks, where scores have a long lower tail of failed runs or pile up below a ceiling. With a stronger model, DeepSeek V4.1 Flash, tasks that are already easy leave a team little room to add. The same model also explains a frontier-model result: at equal dollar cost, Qwen3.8-27B teams beat GPT-5.6-sol on 4 of 8 tasks, because GPT-5.6-sol scores high but varies little from run to run, so sampling it more adds little.

Goes deeper in: Agent Engineering → Agent Teams → Parallel Agents & Voting