CARE-PO is a reinforcement-learning recipe for tool-using LLM agents. It rewards correct answers that use fewer tool calls, measuring calls against a **per-task reference budget** fixed before training, and it trains two behaviours, selected by a low or high effort tag in the prompt.

Why do efficiency rewards make agents stop calling tools?

If calls are priced against a bar computed from the model's own recent attempts, the bar drops whenever the model gets shorter, so the cheapest score keeps moving toward zero calls. GRPO also loses its signal when every attempt in a group makes the same number of calls. In the paper's 27B baseline, the average fell to **0.01 calls per attempt**.

How does CARE-PO stop an agent from skipping tool calls?

Two ways. Evidence the task needs but the attempt never gathered is added to its cost, so a call-free attempt is charged a full budget even when it guesses right. And each training group includes the task's original successful solution as a floor, which pulls the model back toward a working path when most attempts fail.

How is the effort tag different from a reasoning-effort setting?

Reasoning-effort settings mostly control thinking tokens. CARE-PO's tag controls external tool calls, and it is built into the training reward, so the low and high modes are learned behaviours. In the paper, putting the tag on an untrained model barely changed its call count.

Agent· 2026-11-11

CARE-PO paper — Task-anchored tool-call budgets — What does it mean?

TL;DR

What is it: CARE-PO (arXiv 2611.12125) is a reinforcement-learning recipe that trains a tool-using agent to make fewer calls by pricing each call against a task-anchored reference budget: a per-task estimate of how many calls the task needs, set before training starts and never changed.
Why it’s needed: Every tool call costs money and latency, and its result stays in the context for every later turn, yet agents trained only on correctness spend calls freely: in the paper's audit of Qwen3.5 runs on the GAIA benchmark, 68% of all calls were judged removable without breaking the answer.
vs previous: Earlier efficiency rewards such as OTC-PO priced calls against the group's own shortest correct attempt, a bar that moves with the policy and can slide toward an agent that issues no calls at all; CARE-PO fixes the bar per task, charges for evidence never gathered, and adds an effort tag so one trained model serves both a low-cost mode and a high-accuracy mode.

Jargon

Tool call: One request from the agent to an outside function, such as a web search, a page fetch or a code run. Its result is appended to the context, so every later turn re-reads it. See Why Tools?.
ReAct loop: The standard agent loop: reason, call a tool, read the result, and repeat until the agent answers. See Inside a Tick.
GRPO: Group Relative Policy Optimization. The model makes a group of attempts at the same task, and each attempt is pushed up or down by how far its score sits from the group average; that push is called the advantage. If every attempt scores the same, the group teaches nothing.
GAIA: A benchmark of long, multi-step questions that need tools such as web search, page reading and code runs. The paper's main results use its 165-question validation split.
Reference budget: CARE-PO's per-task estimate of how many calls a task needs. A judge model shrinks a known successful solution to its essential steps once, before training, and the number then stays fixed.
Evidence coverage: Each training task carries a checklist of facts its answer depends on. Coverage is the share of that checklist the attempt's tool results actually contain.
Effort tag: A short marker in the prompt, low or high, that selects which of the two trained behaviours the model uses: fewer calls, or more calls for accuracy.
Operating point: One pair of numbers a trained agent delivers: its success rate and its average calls per task. Earlier recipes give one per training run; CARE-PO gives two from one set of weights.

The news. On November 9, 2026, researchers from Shenzhen University and collaborating labs posted CARE-PO (Controllable Anchored Relative Efficiency Policy Optimization). Trained on Qwen3.5 models at 4B, 9B and 27B parameters, it reports that on GAIA's 165-question validation split, five of six operating points raise success rate and cut tool calls at the same time; the sixth is statistically indistinguishable from its base model in success rate while making 58.6% fewer calls. On four held-out multi-hop question-answering benchmarks, the result depends on model size. Read the paper →

Picture a courier team paid a bonus for short routes. If the bonus is pegged to the shortest route anyone on the team drove today, the bar moves every time the team gets faster. Drivers who skip stops look best, the next day's bar is shorter still, and the end state is a van that never leaves the depot. That is the failure the paper names in efficiency rewards whose anchor is computed from the policy's own rollouts: the anchor moves with the policy, and training can drift toward an agent that issues no tool call at all. OTC-PO, the closest earlier method, prices calls against the fewest calls any correct attempt in the group used. In the paper's 27B run, OTC-PO's average fell to 0.01 calls per attempt after training step 300.

CARE-PO replaces the moving bar with a posted route card. Before training, a judge model takes a known successful solution for each task, cuts it down to the essential steps, and records the length as the task's reference budget. Because the budget is fixed per task, the demand on the call count does not tighten as the agent improves, and a hard task with a budget of 10 makes each call cheaper than an easy task with a budget of 3. The cost of an attempt is counted in budgets spent, not raw calls, and a repeated call (same tool, same arguments as an earlier one) is charged twice.

A posted card alone still lets a driver save miles by skipping deliveries, so CARE-PO counts a missed package as extra miles. Each task also carries an evidence checklist, and the share of it an attempt never gathered is added to its cost. An attempt that calls nothing gathers nothing, so it is charged a full budget even when it guesses correctly. The second guard is on the GRPO side, where scores only mean something relative to the group: if every attempt in a group makes the same number of calls, the price of a call no longer separates them. So each group also receives the task's original successful solution, the veteran's logged route, with its advantage clipped at zero. In most groups it scores below the average and adds no gradient; it only pulls the model back to a working path when most of the group fails and the group average drops below it.

The last piece is the sticker on the order. Most agent stacks pick a cost-versus-accuracy point at training time, one set of weights per point; the paper argues that how many calls a deployment is willing to spend is a deployment decision. CARE-PO writes the tier into the reward: with a price factor of 1.5, a call at the low tier costs 1.5² = 2.25 times what it costs at the high tier, and the high tier also gives more partial credit to failed attempts that gathered evidence. Tiers are mixed inside each training group (none : low : high = 0.2 : 0.4 : 0.4), so the model learns what each tag means for the same task. The two modes come from training, not from the prompt: adding the low tag to the untrained 9B model cut calls only from 8.26 to 7.01 and lowered success to 37.0%, while the trained low mode reached 3.42 calls at 39.8%. This is the tool-call version of the reasoning-effort setting many model APIs already expose for thinking tokens.

A worked example, using the paper's constants and an illustrative budget. The efficiency factor is U = 1 + 0.5 · tanh(γ · (1 − u)), where tanh is a smooth S-shaped curve that stays between −1 and 1, so the factor stays between 0.5 and 1.5; u is charged calls divided by the reference budget, plus the share of evidence missed, and γ is 1.5 at the low tier and 1/1.5 ≈ 0.67 at the high tier. Take a task with a reference budget of 4 calls (illustrative) and an attempt that answers correctly with full evidence coverage. In 2 calls, u = 0.5, and the factor is about 1.32 at the low tier but only 1.16 at the high tier, so the low tag rewards saving more. In 8 calls, u = 2, and the factor drops to about 0.55 low and 0.71 high, so the low tag also punishes overspending harder. At exactly 4 calls, u = 1 and both tiers give 1.00. Now the shortcut: a correct answer with zero calls has u = 0 + 1 = 1 because no evidence was gathered, so it also scores 1.00, the same as spending the whole budget, not the top score a raw call count would give it. A wrong zero-call answer has no evidence and earns no partial credit, so it scores 0.

Method (Qwen3.5-9B, GAIA-165)	Where it acts	Success rate	Avg. calls	Source
Untrained ReAct	none	40.2%	8.26	Table 1
Budget in the prompt	inference only	35.4%	8.50	Table 1
DEPO	offline preference training	34.7%	7.54	Table 1
OTC-PO	online RL, anchor from the group	40.6%	5.01	Table 1
CARE-PO, low tag	online RL, fixed per-task budget	39.8%	3.42	Table 1
CARE-PO, high tag	same weights as the row above	43.8%	4.44	Table 1

Where the saved calls come from matters for anyone paying the bill. A call's result stays in the context and is re-processed on every later turn, which is why the cost profile of an agent grows faster than its call count. In the paper's 9B analysis, more than half of the reduction came from attempts that used to run into the 20-call cap, almost all of which ended in a wrong answer: the share of capped attempts fell from 15.6% to 1.4% (low tag) and 3.4% (high tag). The savings are not only from giving up early, either: on the 37 questions that both the untrained model and the low mode answered correctly in every run, calls still fell by 57.5%.

The limits are stated plainly in the paper. On the four held-out multi-hop benchmarks, where most questions need only two hops, the 9B model trades success rate for fewer calls in both modes, while 4B and 27B hold their success rate. The method needs an automatically gradable answer and a known successful solution for every training task, and it was tested only on question answering with one model family. So treat CARE-PO as a recipe for agents with checkable answers and expensive tools, not yet as a general fix for open-ended agent work.

Goes deeper in: Agent Engineering → Cost & Latency → The Cost Profile of an Agent

Related explainers

Token Budgets: affine-typed budget ownership — enforcing a spending budget at run time, where CARE-PO trains the model to spend less
TokenCast: segment-level token forecasting — predicting how context growth turns into cost before an agent runs
Tev-Gym: progress-potential reward shaping — another way to give a long-horizon agent a reward signal before the final answer
CacheRL: cached rollouts for agent RL — making the RL training of tool-calling agents itself cheaper

Continue in trackCost & Latency: where an agent's tokens, calls and seconds actually go

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based