TeamGym paper — Distributed pivotal information in agent teams — What does it mean?
The news. On October 27, 2026, a research team released TeamGym, a gym for evaluating and training language-model teams on decisions built from legal, finance, medical and software-engineering corpora. Across eight models and 12 source corpora, splitting the needed facts across participants lowered success by 22.9 percentage points on average, and a further 19.9 points when participants had to retrieve the facts themselves. Read the paper →
Picture a hospital case-review meeting. The right treatment is obvious to anyone who has seen the lab result and the X-ray, but one doctor holds only the lab result and another holds only the X-ray. Each doctor can be an excellent diagnostician alone and the meeting can still reach the wrong answer, because the answer depends on someone saying their fact out loud. TeamGym turns that picture into a measurement. A pivotal information unit is the one fact a doctor holds, and the question is how many of them reach the person who decides.
The benchmark builds each problem from a real corpus (for example the CUAD contracts set, SEC filings, biomedical evidence, or repository code questions). It takes a reference decision, collects candidate evidence units, and keeps the ones that pass a complete-set check (a check that all candidate facts together support the reference decision) and a leave-one-out check (removing any one leaves it unsupported). Those survivors are the PIUs. Because the PIUs stay fixed while the benchmark changes only how agents receive them, the score changes can be read as effects of the team setup rather than of an easier or harder problem. The routing is the same coordination question the course raises in coordination costs: every extra agent adds a hand-off where a fact can be dropped.
| Condition (same problem, same PIUs) | Who holds the PIUs | Effect on success rate |
|---|---|---|
| Single-Agent All-PIU | One model, given every PIU directly | baseline |
| Multi-Agent PIU-Only | One PIU per participant, given directly | −22.9 points vs baseline (average over 8 models, 12 corpora) = the communication gap |
| Multi-Agent Full-Context | One PIU per participant, buried in source material | a further −19.9 points (same average) = the retrieval gap |
The two gaps answer two different questions. The communication gap asks whether a member who has a fact will share it; the retrieval gap asks whether a member will find the fact in the first place. The paper reports that strong solo performance does not predict team performance: a model that does well with all the PIUs in hand can still do badly as one of several holders.
The paper also shows that how the conversation is run changes which facts surface. When one participant holds all but one of the PIUs, a round-robin structure beat a coordinator-selected speaker by 31.4 points on 59 finance problems with two Qwen3 models, against 17.8 points when each participant held one PIU, a 13.6-point interaction. The cause was narrow: the coordinator's query to a participant could ask for only part of that participant's evidence, even though 99.2% of the required participants did speak. Adding public category labels for the participant's PIUs and an instruction to report on each cut the measured interaction from 15.5 to 5.9 points in a replay. This is the hand-off problem in miniature: an ask that is too narrow loses information that the receiver actually had.
Now the worked example. Hold one problem set fixed and suppose a single model that sees every PIU succeeds on 80% of instances (illustrative; the paper reports average gaps, not this baseline). Splitting the PIUs one per participant removes 22.9 points, leaving 57.1%. Making the team find those PIUs in source material removes 19.9 more, leaving 37.2%. The same facts and the same models would then succeed less than half as often (37.2 / 80 = 46.5%) once the facts are split and must be found. The 42.8-point total is the sum of the two gaps, so a fix that only improves messaging can recover at most the first 22.9 points.
The paper also trains on this signal. Using reinforcement learning with only the final decision as reward, a Qwen2.5-7B coordinator-and-participant setup on contract problems rose from a 12.14 LLM-judged score (a score assigned by an AI evaluator) to 16.84 after training the participants, and to 23.33 after also training the coordinator. A shared participant policy on finance problems rose from 21.85 to 41.08 on validation, and the share of episodes where every participant spoke rose from 61.7% to 86.7%. Nobody rewarded speaking up; participation rose because the final-decision reward required it. Three limits apply: scores are LLM-judged, the problems are built from fixed corpora, and the training runs are small, so the numbers show direction, not a production guarantee.
Goes deeper in: Agent Engineering → Agent Teams → Coordination Costs
Related explainers
- Planning-only delegation in coding-agent teams — asks where a team's workers should be placed, while this article asks whether the facts they hold reach the decider.
- Handoff auditing: introduction vs propagation — another look at where information is lost between agents.