EnvOpt paper — Optimizing the environment, not the harness — What does it mean?
The news. On November 5, 2026, Alex Dimakis posted EnvOpt on arXiv, a pipeline that optimizes the environment an agent runs in. On AutomationBench it cut Claude Haiku 4.5 cost by 72.7% while raising pass rate by 14.34%, and the same environment cut Opus 5 cost by 68.1%. Read the paper →
Picture a clerk who fetches items for orders from a stockroom. Fire the clerk and hire a better one, or rewrite the clerk's training manual, and you are optimizing the model or the harness. There is a third lever the paper points at: the stockroom itself. If every item sits in one pile with no labels, even a great clerk makes many trips, opens many boxes and re-reads the same shelves. Put up an aisle map, an index and a few labeled shelves, and the same clerk finishes the same orders with fewer trips.
That is the artifact EnvOpt searches for. The paper writes an environment as E(I): I is the information the tasks need, and E is the representation of that information. A Slack history behind an MCP server and the same history exported as text files hold the same I but are different E. The goal is the E that minimizes the target agent's cost over the task distribution, for a fixed model and a fixed harness.
The loop in the diagram runs for N steps. At each step the research agent gets a disjoint subset of the training tasks (to prevent overfitting), the environment from the previous step, and permission to run the target agent. It keeps an edit only when cost falls and it judges accuracy preserved, using proxies such as answer consistency, because training tasks come with no verifiers. After each step the candidate environment is scored on a separate validation set with verifiers, and the run returns the previous environment if cost rose or accuracy fell by more than a tolerance. Any change the research agent makes to the harness is discarded.
| Strategy | What the research agent built | Outcome |
|---|---|---|
| Accepted: cheatsheet | A list of the API endpoints the agent needs, so it stops searching for them each task | Fewer discovery calls |
| Accepted: index | One text file listing paths, sizes and types of all needed files (Workspace-Bench) | Search one file instead of crawling the filesystem |
| Accepted: database | An SQLite database of table locations and which script writes which file (Data-Eng-Bench) | Locate information without browsing thousands of files |
| Rejected: one giant file | A single file of all possible column values | Cost rose: the agent saw how much it had not explored and made more tool calls |
| Rejected: loose search tool | A file-search tool returning too many files | Cost rose: extra results invited extra reading |
| Rejected: narrow test check | A repository test that fit only some training tasks | Failed on later tasks; the agent spent much more time bypassing it |
The rejected rows show the limit of the metaphor. A shelf that looks organized can still make the clerk work harder: one huge file told the agent how much it had not yet seen, and a search tool that returned too much gave it more to read. Only trial runs against the real agent separate a helpful layout from a harmful one, which is why EnvOpt searches by running tasks rather than by applying a layout rule.
Most of the saving comes from input tokens spent on tool results (environment feedback), plus fewer tool calls on the output side. That matches the intuition from the Cost Profile of an Agent step: an agent re-reads what its tools return on every turn, so shrinking and sharpening those results pays on every later turn. A tool-only ablation on AutomationBench, which kept the original environment but added the new tools, improved cost and pass rate by less than changing both tools and environment. That gap is the headroom beyond harness engineering the paper claims.
A worked example shows when this pays. Take the paper's reported setup for Claude Opus 4.8 on AutomationBench: an offline optimization that costs around $150 and cuts running cost by 77.5%. Hold two variables fixed that the paper does not give for one task, so treat them as illustrative: $0.50 per task (illustrative) before optimization, and the saving applying evenly to every task. Each task then saves $0.50 × 0.775 = $0.3875, so the $150 is recovered after about 150 / 0.3875 ≈ 387 tasks. Over 1,000 tasks the baseline costs $500, the optimized run $112.50, and the net saving after paying for EnvOpt is $387.50 − $150 = $237.50. The paper's own estimate, assuming 1,000 unseen tasks, is that frontier agents at scale save more than ten times the optimization cost; the smaller illustrative ratio here comes from the low per-task cost chosen.
Three limits come from the paper itself. First, the method fits stationary environments: material that stays the same or similar across many tasks. An environment that changes every task has no stable layout to optimize. Second, a bad early edit can hide for many steps: in Data-Eng-Bench the research agent wrote a SQL statement splitter that broke on semicolons inside quoted text, and the flaw went unnoticed until step 5, inflating cost through the four steps in between because the target agent blamed its own model. Third, the paper reports the research agent sometimes used unreliable proxies such as answer length, which hurt held-out performance, so the validation set with real verifiers stays necessary for auditable results. The authors note that running three training steps without validation looked safe across their five benchmarks, but call that an empirical claim only.
Goes deeper in: AI Agents → Context Engineering → Context as Scarce Resource — and for the cost side, Agent Engineering → Cost & Latency → The Cost Profile of an Agent.
Related explainers
- HarnessBridge, a learned agent harness — the harness axis that EnvOpt deliberately leaves untouched.
- The agent meta-harness — another view of the harness layer that sits next to the environment.