What is EdgeBench's environment-learning scaling law?

It is an empirical curve fit by the EdgeBench benchmark that relates how much an agent has learned to how long it has been interacting with its task environment after deployment. The curve is log-sigmoid — slow at first, fast in the middle, then leveling off — and EdgeBench reports it fits with R²=0.998 across 134 real-world tasks and roughly 38,000 hours of agent interaction. Separately, on a much longer clock, the paper reports that agent learning speed roughly doubles every three months — a trend distinct from the shape of the within-task curve.

Why measure agents learning after deployment instead of one-shot accuracy?

Because one-shot accuracy grades an agent like an orientation packet — a single fixed test scored once — and misses the part that actually matters in production: how fast the agent improves by doing the real job. EdgeBench runs each task for at least 12 hours of continuous operation and grades on multi-level feedback the whole way, so it captures the learning curve rather than a single snapshot. That distinction matters because an agent's competence builds along a curve, not in one step, and an early one-shot grade cannot see that shape.

What does 'learning speed doubles every three months' mean in practice?

It is a reported trend that the pace of agent learning is roughly doubling every three months: successive systems get about twice as fast at learning from real-world interaction each quarter, per the paper. It is distinct from the within-task log-sigmoid curve — one describes how a single agent improves as it interacts, the other describes how quickly that learning ability is itself improving over time. It is a fit on the paper's own tasks, so whether the exact ~3-month figure transfers to your stack is something to measure rather than assume.

EdgeBench measures scaling laws for agents learning in the wild — Environment-learning scaling law

Jargon

Environment learning: Improvement an agent gains by interacting with its task environment after deployment — using tools, reading results, getting corrected — rather than during pretraining. EdgeBench is built to measure this on-the-job gain.
Scaling law: An empirical curve that predicts an outcome from one quantity. Here the outcome is how much an agent has learned, and the quantity is how long it has been interacting. A tight scaling law means the curve explains almost all of the variation.
Log-sigmoid curve: An S-shaped curve on a log time axis: slow at first, fast in the middle, then leveling off. It is the shape EdgeBench fits to environment learning — the mathematical version of a new hire's ramp.
R² (fit quality): The fraction of the variation the curve explains, from 0 (predicts nothing) to 1 (predicts everything). EdgeBench reports R² = 0.998, so the curve is close to a law rather than a loose trend.
Long-horizon task: A task that takes an agent a long, unbroken stretch of work to finish — EdgeBench requires at least 12 hours of continuous operation each, so the ramp has room to show up. Related to compounding errors over many steps.
Multi-level feedback: Signal the agent receives throughout the task, at several granularities — a final result, intermediate checks, and step-by-step corrections — instead of a single pass/fail at the end. It is what environment learning learns from.

The news. On July 6, 2026, researchers released EdgeBench, a benchmark that studies agents learning after deployment instead of during pretraining. Across 134 real-world tasks and roughly 38,000 hours of agent interaction, they fit a log-sigmoid scaling law for environment learning with R² = 0.998, and release 51 tasks plus the evaluation framework. Each task runs at least 12 hours of continuous operation and is graded on rich feedback the whole way, not only the final answer. Their headline reading: "agent learning speed roughly doubles every three months." Read the paper →

Think about how a new hire actually gets good. The orientation packet on day one barely moves the needle; what makes them useful is doing the real work, shift after shift — closing tickets, reading the results, getting corrected by a manager and by reality. Environment learning is that on-the-job improvement, and EdgeBench is the study that finally measures it for AI agents instead of grading the orientation packet. Most agent benchmarks grade the packet: one fixed test, scored once at final-answer time. EdgeBench drops the agent into a real job and watches it ramp.

To see a ramp you have to watch long enough for one to happen. So each EdgeBench task runs for at least 12 hours of continuous operation — ultra-long-horizon work spanning scientific discovery, software engineering, combinatorial optimization, professional knowledge work, formal mathematics, and games. Across 134 such tasks that adds up to roughly 38,000 hours of interaction. And the agent is judged on multi-level feedback the whole way — the intermediate checks, the results coming back, the corrections along the route — not a single grade at the end. That is the difference between an online eval that watches the live job and an offline quiz taken once.

Now plot how much each agent has learned against how long it has been on the job, and the messy cloud of runs collapses onto one clean curve — a log-sigmoid scaling law that fits with R² = 0.998. That shape is the new hire's ramp made precise: slow while they learn the ropes, fast once it clicks, then leveling off near mastery. A fit that close to a perfect 1.0 means the curve is an exceptionally tight description of these runs — how long an agent has been interacting explains almost all of the variation in how much it has learned.

Where the ramp earns its keep

Two of the paper's numbers already tell a story when you divide them. 134 tasks over roughly 38,000 hours of interaction is about 284 hours of work per task on average — more than 23× the 12-hour floor. These are not quiz questions; each runs at least 12 hours nonstop and, on average, far more interaction than that, so a single final grade throws away almost everything that happened along the way. Separately from the shape of any one task's curve, the paper reports a second trend on a much longer clock: agent learning speed roughly doubles every three months — a compounding pace where each three-month step is about twice the learner the last was, 1 → 2 → 4 → 8 across three doublings (illustrative decomposition of the reported ~3-month doubling; the exact per-period multipliers are not published). Neither the within-task learning curve nor this quickening pace shows up in a one-shot accuracy score, which is the gap EdgeBench is built to close.

Way to grade an agent	What it captures	Long-horizon signal
One-shot task accuracy (most benchmarks)	a single score at final-answer time	none — the ramp is invisible
Leaderboard snapshot	rank on a fixed test set at one moment	can mislead under distribution shift
EdgeBench environment learning	a learning curve over ≥12h of continuous work, 134 tasks (paper)	a log-sigmoid learning curve, R² = 0.998 (paper)

Because the signal is a curve over interaction time rather than a point, it changes what you can measure: you can compare whole learning trajectories instead of relying only on snapshot accuracy, and tell a slow-learning environment from a fast one. The usual caveats apply — this is a scaling-law fit on the paper's own 134 tasks, a tight correlation is not a guaranteed control knob, and whether the ~3-month doubling transfers to your stack is exactly the kind of thing to measure in shadow rather than assume. The shift EdgeBench is really arguing for is one of instrument: grade agents on the shape of their learning curve, not a snapshot of their accuracy.

Goes deeper in: AI Agents → Evals & Diagnostics → Pass/Fail vs Score

Continue in trackProduction Evals: online vs offline, and why watching the live job matters

Related explainers

Effective Feedback Compute (EFC) — the other agent scaling law: success tracks feedback quality, the fuel this on-the-job ramp runs on
Agents-A1 — scaling the horizon, not the parameters — a different agent scaling axis: how long a task the agent can sustain, complementary to how fast it learns
Agent leaderboards & predictive validity — why a one-moment leaderboard snapshot misleads, the failure EdgeBench's learning curve is built to avoid
AutoLab — iterative experiment-loop evaluation — another long-horizon benchmark that grades the loop, not a single answer

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based