What is observation-effect separation?

It is the practice of recording two histories for an agent workflow instead of one: what actually happened in the outside world, and what the runtime observed about it. An effect either occurred or it did not, while `unknown` is a state of the runtime rather than of the world. Keeping them apart is what lets you name the failures that come from acting under uncertainty.

Agent retry, commit and recovery logic is written against observations, but the consequences land in the world. When a tool call's outcome is unconfirmed, re-issuing risks a duplicate effect, committing risks a missing one, and compensating risks refunding something that never happened. All three look like reasonable engineering and all three can be wrong.

Can a wrapper around a tool fix this?

Only partly. The paper identifies four points where black-box invocation alone cannot give a general guarantee: resolving an ambiguous outcome, repairing a conflict between non-commuting irreversible effects, retracting an effect an outside party already acted on, and releasing several irreversible effects across tools atomically. The remedies differ — authoritative tool participation for the first and the last, shared mediation for the conflict, and visibility control for the open-world reaction.

How does it relate to idempotency?

Idempotency is what makes the retry branch safe, because an idempotent re-issue that returns the original outcome converges on one authoritative answer. It is only one of three routes out of an unknown outcome, though — a status endpoint and a durable acknowledgment are the others — and it is the only one of the eight required boundary capabilities that current MCP annotations express at all, as a coarse hint.

What did the MCP census find?

Across 98,291 tools from the official registry snapshot of 27 July 2026, the four annotation fields are widely emitted — 74.0% of tools serialize at least one and 61.7% all four — but coarsely used, with a single signature covering 39.9% of tools. None of the fields carries an idempotency key, a status endpoint, a compensation contract or a commutativity rule.

Agent· 2026-09-15

A tool call can succeed while the workflow fails — Observation-effect separation — What does it mean?

TL;DR

What is it: The paper When Tool Calls Succeed but Workflows Fail (arXiv 2609.15397, 14 September 2026) introduces an effect-history model: a way of recording what a tool actually did in the outside world separately from what the agent runtime managed to observe about it.
Why it’s needed: Long-running agent workflows push real effects out into the world — money moves, a seat is booked, an email leaves — while the runtime's retry, commit and recovery logic is written against its own observations, so every gap between the two histories is a place where the world and the workflow silently disagree.
vs previous: Classical transaction processing assumes an in-doubt outcome is temporary, because every participant runs a recovery protocol until the outcome resolves; a third-party tool runs no such protocol, so an unknown outcome simply stays unknown, and the paper catalogs the eight anomalies that follow from acting anyway.

Jargon

Externalize: A tool call externalizes when it changes durable state outside the agent — the money moves, the seat is held, the email leaves the building. Once an effect is externalized it exists whether or not the runtime ever hears about it.
Unknown outcome: The runtime has no authoritative answer about whether an effect happened. This is not the same as failure: in the world the question is already settled, and only the runtime is in the dark.
Idempotent: An operation is idempotent when issuing it twice leaves the same state as issuing it once. delete is idempotent; increment is not. It is the property that makes a blind retry safe — see Idempotency — Safe to Re-run.
Compensation: A second operation that undoes a first one — a refund against a charge. A compensation is not a rollback: it externalizes in its own right, so issuing one against an unconfirmed effect may cancel nothing and simply add a second effect of its own.
Exactly-once barrier: The classical result that over an unreliable channel a caller cannot learn in bounded time whether its request took effect. It is why the first three anomalies cannot all be avoided by the caller alone.
Commutativity: Two operations commute when running them in either order leaves the same result. It is a property of a pair of calls on the same resource, not of one call on its own: two increments commute, an increment and a set do not.
MCP annotations: The four hint fields an MCP tool may declare about itself: readOnlyHint, destructiveHint, idempotentHint and openWorldHint. They describe the shape of a call, not its transactional semantics. Related: tool design as context engineering.
Open-world effect: An effect an outside party can observe and react to. Once a supplier has read the purchase order and started shipping, retracting the order does not retract the shipment.

The news. On 14 September 2026, Artem Trofimov and Boris Novikov posted When Tool Calls Succeed but Workflows Fail: Anomalies at the Agent-Tool Boundary. The paper argues that agent reliability needs an explicit model of external effects rather than only of tool responses. It contributes an effect-history model separating events in the world from the runtime's observations of them, a catalog of eight recurring external-effect anomalies, the boundary capability each one needs in order to be excluded, and four points where black-box tool invocation alone cannot give a general guarantee. It then measures how much of that is expressible in the Model Context Protocol's standard annotation vocabulary across 98,291 registered tools. Read the paper →

Stand at the terminal for a moment. You tapped, the screen went blank, and no receipt came out. The terminal's silence is not a failure — it is the absence of an answer, and those are different states. Somewhere behind the counter the bank has already decided: either the money moved or it did not. The decision is final and it is not yours. What you are missing is not the outcome but the news of it.

An agent runtime stands in exactly that position after every tool call that does not come back cleanly. The paper's move is to stop writing one history and start writing two. One history records the world: an attempt was made, an effect was externalized, a compensating action ran. The other records the runtime's knowledge: this attempt was confirmed, that one failed, this third one is unknown. Ordinary read-write histories in a database collapse these together, because a database owns both sides of the line. An agent does not own the bank, and a tool it calls over MCP is a stranger's service that may itself be a workflow of unknown depth. When that hidden depth shows through — a timeout inside the provider's own multi-step process — the caller is left holding an outcome it cannot confirm.

The two histories also explain why the boundary matters more than the tool. Build the agent and the tools together and you can declare each operation's properties by construction. Reach across a shared interface and you see only what the interface chooses to carry. The paper works with five properties of a tool call that a runtime actually needs: whether it is idempotent, whether it can be inverted, when and whether its effect can be held back before release, whether it is deterministic, and whether it commutes with other calls on the same resource. These do not reduce to one reversible-or-not scale. increment commutes with another increment but is not idempotent; delete is idempotent but does not commute with a write.

attemptruntime issues pay_invoice

externalizethe charge lands in the world

observeno acknowledgment returns

resolvethe workflow must choose, blind

That last box is where the catalog begins. The runtime has to resolve the workflow, and against an unconfirmed charge the paper narrows the safety-relevant reactions to three. It can also simply wait — but waiting creates no new effect history until one of the three is taken, and it buys that by giving up resolution. It can re-issue the call, on the theory that nothing happened — and if something did, the invoice is paid twice (A1, Duplicated Effect). It can commit the workflow, on the theory that the charge landed — and if it did not, the order is marked paid with no payment behind it (A2, Missing Committed Effect). Or it can send a compensating refund, on the theory that the charge must be cleaned up — and if the charge never happened, it has just refunded money nobody took (A3, Orphaned Compensation). All three reactions are reasonable, and each one risks a different anomaly, because the runtime is choosing without evidence. A3 is the sharpest of them: it counts as a violation at the moment the compensation is issued, not at the end. Even where the balance happens to come out right, the action was taken without authoritative evidence.

Grouping them like that is the paper's first useful result: A1, A2 and A3 are not three bugs, they are one missing capability seen from three angles. The runtime does not need better retry logic. It needs the tool to offer a way out of unknown — a durable acknowledgment, a status endpoint it can poll, or an idempotent re-issue that returns the original outcome instead of producing a second one. Which of those the tool offers determines which branch of your retry policy is actually safe to take.

Anomaly	The forbidden pattern	Expressible in MCP annotations today?
A1 Duplicated	One logical operation externalizes twice.	Limited — an idempotence hint only
A2 Missing	The workflow commits without a required effect.	No
A3 Orphaned	A compensation is issued under an unknown outcome.	No
A4 Residue	An aborted workflow leaves a surviving effect.	No
A5 Premature	An effect that may not survive is released before the workflow resolves.	No
A6 Contaminated	A committed effect depends on an effect that did not survive.	No
A7 Conflicting	Non-commuting effects from independent executions land unordered.	No — externality only
A8 Phantom	An outside party already acted on an effect that was later compensated.	No

The three families behind those rows are worth holding onto. A1 to A3 are uncertainty anomalies: the runtime cannot confirm an outcome and must act anyway. A4 to A6 are workflow anomalies, about one execution's own shape — an effect surviving an abort, releasing before resolution, or being depended on by something that then commits. A7 and A8 are interaction anomalies, where an effect meets something outside its own execution: another workflow writing the same resource, or an outside actor who sees the effect and reacts. Not every row is a final-state violation: A3 is defined at the moment of action, and A5 and A7 are preventive, judged against a profile that forbids relying on outcomes the boundary never established. The catalog is also coverage-oriented rather than complete by theorem — the authors conjecture that every loss of transactional control their vocabulary can represent falls into one of the three families, and leave the proof to future work. Four guarantee boundaries mark where a caller wrapping a black-box tool runs out of room, no matter how careful the wrapper is — you cannot resolve an ambiguous outcome without tool-side participation, you cannot repair a conflict between two non-commuting irreversible effects after the fact, you cannot un-ring a bell an outside party has already heard, and you cannot release several irreversible effects on different tools atomically from above the tool layer. The first of those is the exactly-once barrier turning up at the agent-tool boundary; the last is the classical atomic-commitment problem, and it needs the tool to offer a two-step commit — one call that stages the effect without releasing it, and a second that releases or discards it — before anyone above the tool layer can offer a guarantee at all. What each boundary needs also differs: authoritative tool participation for the first and the last, a mediator or an ordered release for the conflict, and visibility control before observation for the open-world reaction. The authors state all four as consequences of their definitions and leave the formal proofs to future work.

How much of this does MCP already say?

The paper's census is the part with the numbers, and it is worth walking rather than summarizing. Take a full snapshot of the official MCP registry from 27 July 2026: 59,625 entries covering 18,688 distinct servers. Of those, 9,454 exposed no remote endpoint and were out of scope, leaving 9,234 remote targets to query; 4,838 of them returned at least one tool, yielding 98,291 tools at a median of 11 per server. That is the denominator — hold it fixed and watch one field.

destructiveHint is the one annotation about risk, and it is set on 65.8% of all tools, which is about 64,675 of them. That number looks like broad coverage. But destructiveHint is meaningless when readOnlyHint is true, and that exact pairing accounts for 52.8% of the census. Filter down to the tools where the field can say anything at all and you are left with 12.9% — roughly 12,680 tools, or about one tool in eight. Narrow again to the tools that actually assert a destructive operation and it is 3.1%, some 3,047 tools, about one in thirty-two. A field that appears to cover two thirds of the registry carries a usable risk classification for one tool in eight.

The shape repeats across the vocabulary. 74.0% of tools serialize at least one of the four fields and 61.7% serialize all four, so emission is not the problem. Read each tool as a signature over the four fields and 66 of the 81 possible signatures appear — but one signature, read-only and non-destructive and idempotent and open-world, covers 39.9% of tools, the next most common is no annotation at all at 26.0%, and the top three together cover 75.8%. The fields are widely emitted and coarsely used. Even fully populated they describe the shape of a call while leaving its transactional semantics unstated: none of them gives an idempotency key, a status endpoint, a compensation contract, or a commutativity rule.

Two honest caveats the authors state and worth repeating. The census measures the reachable remote subset — 4,318 targets failed to connect and package-only servers were never probed — so unreachable servers may differ. And it reads the boundary as declared, not as implementations behave: a tool may deduplicate internally and never say so. That safety is real, and it is still unusable to a runtime that can only see the interface. Idempotency is not a nice-to-have on the tool side — an idempotent re-issue that returns the original outcome is one of only three routes out of unknown, and it is the only one the interface can even gesture at today.

Goes deeper in: Agent Engineering → Production Harness Architecture → Idempotency — Safe to Re-run

Related explainers

LedgerAgent's pre-tool-call policy validation — the other half of the boundary: checking a call against a structured state ledger before it externalizes anything.
ToolFailBench's tool-use failure taxonomy — failures on the model's side of the line, where the agent skips, ignores or fabricates a call.
MCP SEP-2663's async task handles — the protocol work that gives a long-running call a handle you can ask about later, which is one route out of unknown.
Tool design as context engineering — how much of an agent's behaviour is decided by the tool interface rather than the model.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based