What is regression control in agent optimization?

Regression control is a guard built into an agent optimizer's loop that biases the optimizer away from updates which improve performance on new tasks by degrading tasks the agent already handled. In the study (arXiv 2607.14004), the optimizer RELAI-VCL biases its search away from shortcut solutions that score well locally but do not generalize — described as an inductive bias against shortcuts that erase earlier gains. It is the mechanism the authors credit for gains that compound over time instead of washing out.

Why don't one-shot optimization gains transfer to real deployment?

Because deployment is continual, not one-shot. You optimize a harness, new tasks arrive, and you optimize again — and an optimizer with no memory of its past wins can take a shortcut that boosts the newest tasks while quietly breaking older ones. The study measures this with a two-phase test (optimize, add new tasks, optimize again) and finds that one-shot benchmark gains do not reliably survive the second round unless regression control is in the loop.

Which optimizer won, and by how much?

RELAI-VCL, the one with regression control, posted the highest lifelong average pass rate on hard Terminal-Bench 2.0 tasks at 76.4%, versus 66.0% for GEPA and 64.6% for Meta Harness (both without a regression guard) and 58.7% for the un-optimized baseline. Under the same task family and budgets, the paper credits regression control for gains that compound instead of washing out.

Agent optimizer study shows regression control compounds gains — Regression control in continual optimization

TL;DR

What is it: A continual-optimization study (arXiv 2607.14004) pits three agent-harness optimizers — GEPA, Meta Harness, and RELAI-VCL — against hard Terminal-Bench 2.0 tasks, and its idea is regression control: a guard inside the optimization loop that biases the optimizer away from updates that improve new tasks by quietly breaking old ones.
Why it’s needed: Real teams never optimize an agent once. They keep re-tuning the harness as new tasks arrive — and without a guard, a tuning round can undo the last one, so a harness that topped a benchmark can quietly regress once new tasks land.
vs previous: A one-shot optimizer maximizes a single benchmark pass and never checks whether the change survives the next batch of tasks. Regression control turns optimization into a continual loop that keeps earlier tasks in view, not just the newest — and the method that uses it posts the top lifelong pass rate, 76.4% versus a 58.7% baseline.

Jargon

Continual (lifelong) optimization: Re-optimizing an agent repeatedly as new tasks keep arriving, instead of tuning it once and freezing it. The realistic setting: you ship a harness, new work shows up, you tune again — over and over.
Agent-harness optimizer: A method that tunes the agent's scaffolding — its prompts, tools, and control flow — rather than the base model's weights. GEPA, Meta Harness, and RELAI-VCL are all harness optimizers, not new models.
Regression control: The paper's guard inside the optimization loop: it biases the optimizer away from updates that raise the score on new tasks by lowering it on tasks the agent already handled — described as an inductive bias against shortcuts that don't generalize.
Shortcut solution: A change that scores well on the tasks right in front of it by exploiting a pattern that doesn't transfer, so it quietly erases gains elsewhere. The optimizer looks like it improved; the agent didn't.
Terminal-Bench 2.0: A benchmark of hard command-line / terminal tasks used to score the agents. The study runs the optimizers on its harder tasks under identical budgets, so any difference comes from the method, not the compute.
Lifelong average pass rate: The headline metric: average success across all tasks, old and new, over the whole continual run — not a single benchmark pass. It is the number that falls apart when an optimizer forgets.
GEPA · Meta Harness · RELAI-VCL: The three optimizers compared. GEPA and Meta Harness optimize without a regression guard; RELAI-VCL is the one that builds regression control into its loop and posts the top lifelong pass rate (76.4%).

The news. On July 15, 2026, a study on continual agent optimization (arXiv 2607.14004) asked whether agent optimizers keep improving once new terminal tasks arrive after the first optimization pass. Comparing GEPA, Meta Harness, and RELAI-VCL on hard Terminal-Bench 2.0 tasks under identical budgets, it found that one-shot benchmark gains do not reliably transfer to continual deployment — and that gains compounded only when regression control was built into the optimization loop. Read the paper →

Picture the strength coach. Every week they add a new lift to your program, and every week they can rewrite the whole plan to get you through it. The fast way to look good on this week's lift is a shortcut — change your form, borrow some momentum, drop the boring accessory work — and the number on the bar goes up today. But the next time you test your squat, it's quietly down. You didn't get stronger; you traded old strength for a new party trick. That trade — a change that boosts the task in front of you while eroding the ones behind you — is exactly what this study measures in agent optimizers.

The paper runs a two-phase test. First it optimizes the agent's harness on one set of Terminal-Bench 2.0 tasks; then it introduces new tasks and spends a second, equal optimization budget. The key move is that GEPA, Meta Harness, and RELAI-VCL are treated as agent-harness optimizers — programs that rewrite the agent's prompts, tools, and control flow — not as static agents. The question is not "who scores highest after one pass" but "whose round-one gains are still there after round two." That is the difference between acing a benchmark and safely improving a harness over time, and it is where one-shot scores start to lie.

RELAI-VCL adds one thing the others lack: a regression guard. It biases the optimizer away from any change that wins new tasks by erasing the wins it already had — in the coach's terms, it keeps the earlier lifts in view, so a shiny new form cue isn't kept if it wrecks them. The authors put it plainly: "Optimization gains compounded only when regression control was built into the optimization loop, providing an inductive bias against shortcut solutions that fail to generalize." It is the same instinct as rollback discipline in a real rollout — never let a "better" version ship if it silently breaks what already worked.

Reading the four numbers

Line the lifelong pass rates up and the pattern is sharp. The un-optimized baseline clears 58.7%. The two optimizers without a regression guard land close together — Meta Harness at 64.6% and GEPA at 66.0%, a modest 6-to-7-point step above baseline. The one with regression control, RELAI-VCL, reaches 76.4% — roughly 10 points clear of the best guardless optimizer. All four run on the same task family under identical budgets, and the paper's stated explanation for the gap is the guard: "gains compounded only when regression control was built into the optimization loop."

Optimizer	Regression control built in?	Lifelong avg pass rate (Terminal-Bench 2.0)
Baseline (no optimization)	—	58.7%
Meta Harness	No	64.6%
GEPA	No	66.0%
RELAI-VCL	Yes	76.4%

Notice where Meta Harness and GEPA land — 64.6% and 66.0%, close together and only a step above baseline, while RELAI-VCL pulls clear at 76.4%. The study frames the takeaway around continual deployment: one-shot benchmark gains do not reliably transfer once new tasks keep arriving — the same compounding-error dynamic that makes long agent rollouts brittle — and in this two-phase test, the method that guards its earlier wins is the one that keeps compounding.

Goes deeper in: Agent Engineering → Production Evals → Eval-Driven Rollout

Related explainers

FAPO — Failure-attribution-gated prompt optimization — also gates a harness optimizer, but on which step caused a failure, where this study gates on whether an update erases old wins
Omnigent — The agent meta-harness — the kind of meta-harness these optimizers rewrite; this study asks whether tuning one once actually holds up
Long-Horizon-Terminal-Bench — Partial-reward threshold — the same Terminal-Bench task family, graded densely instead of pass/fail
HarnessBridge — Learned agent harness vs hand-engineered — what an "agent-harness optimizer" is actually rewriting under the hood

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based