BOTTLED — Amortizing an agent into a reusable artifact — What does it mean?

The news. On October 6, 2026, researchers from the University of Tübingen and collaborators posted Agent in a Bottle, which introduces BOTTLED: an agent gets an entire unlabelled workload and must return an answer for every item within a 10-hour, 5-million-token budget. Read the paper →

Picture a restaurant that gets millions of orders for the same kind of dish. The zero-shot approach hires the master chef to cook every plate: quality is high, but the bill grows with every order. Bottling asks the chef to spend one shift writing a recipe card and training a line cook, then step away. In the paper's formula the total cost is a one-time build cost plus the cost of running the artifact on every item, and in nearly all the runs the artifact spent zero further model-API tokens because it was self-contained.

The agent works inside a harness with a shell, Python, one A100 GPU and access to its own model's API. It sees the full workload at once and chooses its own route: write rules, label a sample and fine-tune a small model, or mix both. Three tasks were used, each with millions of instances: attribute extraction (MAVE, about 4.77 million pairs), query-product relevance (ESCI, about 2.62 million) and AI-text detection (RAID, about 5.62 million). The agent has to judge whether an upfront investment will pay back across the batch before it spends the budget.

The headline result is a mismatch. Two models with the same zero-shot score can bottle very differently, because bottling needs different skills: sampling and labelling data, validating a pipeline, and stopping on time. Across 10 models and 3 tasks, 48 of 60 bottling runs scored below the lower bound of their own model's zero-shot 95% confidence interval, and 31 of 60 runs fell below the stronger of two naive distillation baselines that spent the same token budget.

Bottling can still pay. On ESCI, the paper reports Opus 5 reaching a mean bottled macro-F1 of 0.499 against 0.609 zero-shot, at a reported cost of $26.28 against an estimated $17,273 for running the full workload zero-shot. That is about 82% of the quality at roughly 657 times lower cost. Against Jev, a model built for cheap repetitive inference, the same bottled result is about 94% of Jev's macro-F1 (0.531) at about a quarter of Jev's projected full-workload cost of $105 (the paper's figure).

A worked example (illustrative, derived from the paper's reported ESCI figures): $17,273 spread over about 2.62 million pairs is roughly $0.0066 per zero-shot call. The bottled run cost $26.28, which buys about 4,000 zero-shot calls. So the recipe card pays for itself once the workload passes roughly 4,000 items, and at 2.62 million items the same card is the 657-times saving, with 0.499 / 0.609 ≈ 82% of the quality. The dollar figures are the paper's own reported costs, computed from a weighted token count (uncached input 1, cached input 0.1, output 2), so they are cost units rather than an invoice; treat the break-even as approximate.

For an engineer, the lesson is to price the repeated call before you scale it. The cost profile of an agent shows where tokens go per run, and result caching is the lightest form of the same idea: reuse work across related requests. BOTTLED goes further and asks the agent to decide whether a reusable artifact is worth building at all. The study covers three narrow tasks and one harness, so it does not show that every workload can be bottled.

Goes deeper in: Agent Engineering → Cost & Latency Engineering → The Cost Profile of an Agent