Ask on Demand — Pull-based peer retrieval — What does it mean?
The news. On October 30, 2026, a team with authors from Beihang University, Tsinghua University and China Telecom AI posted Ask on Demand: Targeted Peer Communication for Parallel Agentic Systems. It tests the idea on two interactive benchmarks, AppWorld and WebShop. Read the paper →
Picture three detectives working the same case in three separate rooms, each with a private notebook. One of them finds that the key witness only works night shifts. That fact is sitting in one notebook, and the detective in the next room who is about to waste a day knocking on daytime doors has no way to see it.
The usual fix is a nightly photocopy: every detective summarizes the day and everyone gets the pile. That adds context and inference cost for every agent, and it has a deeper flaw (a simplified picture of methods that differ in detail, as the table below shows). A summary is written before anyone knows what a teammate will need. A detail that looked unimportant on Monday becomes decisive on Thursday, and by then it was left out of the summary. The paper also notes that heavy sharing can push independent explorers toward the same lead, which removes the benefit of having three detectives.
AOD replaces the photocopy with a phone call to a records desk. When a detective hits a specific gap, she writes down a short request, and a reply to that request, not a pile of everyone's summaries, comes back. In a parallel team this is one extra tool, ask_peer, and four moves:
- Request. The same model call that chooses
ask_peeralso writes the request: what fact is missing, what the agent has seen so far, and what it plans to do depending on the answer. - Gate. A small 0.6B-parameter model reads the request and the asker's local context and says approve or deny, before any peer is consulted. A denied request does not block later ones.
- Peer selection. An approved request is matched against the other agents' completed records with BM25, and only the single peer with the best-scoring passage is consulted. If nothing scores above zero, AOD returns an "insufficient" reply instead of guessing.
- Evidence-grounded answer. That peer's frozen model answers from the retrieved excerpts and cites them with exact text, without taking new actions in its own environment. This is ordinary retrieval-augmented generation, with the other agents' histories as the document store.
| Method | What is decided | When | Risk |
|---|---|---|---|
| LTS (adapted) | Which intermediate steps are admitted to a shared memory | When the step is produced | A step that looks minor now may be the one a teammate needs later |
| AgentFugue (adapted) | Compresses finished segments into episode notes, raw text kept in an external hub | When the segment completes, retrieved later by intent | Notes can omit details; compression also changes the agent's own context, which confounds cost comparisons |
| Ask on Demand | Which single peer to ask, and whether to ask at all | When the asker has a specific need | A bad or unneeded request costs tokens; the gate exists to filter these |
The hard part is knowing when asking is worth it. A question can be perfectly answerable and still not help the team. AOD teaches the gate from outcomes. It saves the full state of the team just before a request, then runs two continuations from that point: one approves the request, one denies it. Both continue to the end, so the paper measures the difference in final team score and in remaining token cost. The first estimate uses 5 pairs, adds 5 more while the result is unstable, and stops at 20.
The label rule is simple: a request that raises the team score is labeled approve even if it costs more; if the score is tied, it is approve only when it lowers the cost. The 0.6B model is fully fine-tuned on these labels, weighted by how many valid pairs backed each one. Only the gate is trained; the task-solving and answering models stay frozen. At inference the gate needs just the request and the asker's local context, with no replays and no reward signal.
Two worked examples show how the pieces compose. First, the label rule with illustrative numbers: a request whose paired runs show a team-score gain of +0.10 and 8,000 extra tokens is labeled approve, because the gain decides it. A request with no score change and +4,000 tokens is labeled deny. A request with no score change and 3,000 fewer tokens is labeled approve.
Second, the paper's own gate ablation, holding the model (Qwen3.5-35B-A3B), the team size (three agents) and the benchmark (AppWorld) fixed. Task Goal Completion is 56.67 when requests are handled autonomously, 57.50 with a prompted gate, and 62.78 with the trained gate. Training the gate alone adds 6.11 points over the autonomous setting, with the same asking and answering code. Against fully independent agents, AOD's gain on this setup is 8.89 points.
There are limits to the evidence. AOD had the best selected performance in three of four model and benchmark settings, and LTS led on 4B AppWorld. It used fewer tokens than PeriodicShare and LTS in all four settings, but AgentFugue used fewer tokens in three, partly because it also compresses the agent's context, so that comparison does not isolate communication cost. The Oracle@3 result also shows the team's final pick of a trajectory is imperfect. This is a result on two benchmarks, not a general guarantee.
If you build agent teams, the practical reading is about timing: make teammates' information retrievable by query, and put a cheap learned filter in front of the call. Compare it with handoffs, where one agent passes everything onward, and with the coordination costs you pay whenever agents talk.
Goes deeper in: Agent Engineering → Agent Teams → Coordination Costs
Related explainers
- Cohere coding-agent teams — Planning-only delegation — another look at what a team should pay for.
- Handoff auditing — what changes when information moves between agents.