Tev-Gym — Progress-potential reward shaping vs terminal-only reward — What does it mean?
The news. On October 13, 2026, researchers at Adobe Research and UCLA published Tev, a recipe for training long-horizon agents. It has two parts: Tev-Gym, 1,170 verified task environments across 7 domains, and PPRS, the dense reward that trains on them. The resulting 27B model, Tev-27B, matches GPT-6 Astra and Claude Opus 5 at Pass@5 on the computing and math-sciences part of Agents' Last Exam (ALE-CM); on the mean graded score at best-of-5 it still trails them (0.54 against 0.63 and 0.60). Read the paper →
Picture a trail up a mountain where the only reward is planting the flag on the summit. Most hikers, at first, turn back long before the top. A guide who only records "summit reached: yes or no" writes the same zero next to every name, whether the hiker stopped at the car park or one hundred metres below the flag.
That is what a terminal reward does to a group of agent rollouts. GRPO scores each rollout in a group against the group average, so a group in which every rollout fails has an advantage of exactly zero for every rollout and contributes nothing to the gradient. If the policy solves a task with probability p and the group has G rollouts, the share of wasted groups is (1 − p)G: the paper computes 66% at p = 0.05 and 92% at p = 0.01 with G = 8. Long tasks put an untrained open model at roughly one in ten or lower, so the compounding errors that make long tasks hard show up in training as a mostly empty batch.
Tev fixes this by putting huts on the trail. Every Tev-Gym environment is built from a latent world, a hidden and complete spec of the task and the correct state after each step, so the environment knows exactly which intermediate states are right. Its progress contract lists checkable milestones (nodes), each with a weight, and a partial scorer grades the agent's workspace on each one. Credit is prerequisite-gated: a node's score is the lowest score among itself and everything it depends on, so a hut counts only if the huts below it are reached. The progress potential is the weighted sum, Φ(s) = Σ wv · c̃v(s), a number from 0 to 1.
Three properties keep Φ from being gamed, and the paper enforces each with a qualification gate: it is anchored (the untouched starting workspace and every gaming probe score 0, and Φ = 1 exactly when the strict verifier accepts), monotone (it rises as nodes complete and falls when finished work is destroyed), and independent (the scorer's code shares nothing with the verifier or the reference solvers). Because the correct states come from the latent world, the paper notes this avoids human step labels, a trained reward model and a language-model judge.
The per-step reward is the height climbed on that leg, rt = Φ(st) − Φ(st−1), and the rollout reward adds it to the summit flag: R(τ) = 1[τ passes] + λ · Φ(sT), with λ = 0.25 in every run. The sum telescopes, so the reward depends only on the final workspace, not on the route. A passing rollout earns 1 + λ, more than any failing one, so finishing always outranks partial progress.
Here is the whole effect on one group, with made-up numbers (illustrative). Four rollouts all fail to submit a passing workspace, ending at Φ = 0.1, 0.3, 0.5 and 0.7. With a terminal reward the rewards are 0, 0, 0, 0: the mean and standard deviation are both 0, so every advantage is 0. With PPRS the rewards are λΦ = 0.025, 0.075, 0.125 and 0.175, with mean 0.1 and standard deviation about 0.056, so the advantages come out near −1.34, −0.45, +0.45 and +1.34. The group now ranks its rollouts by progress and the gradient points toward the ones that climbed highest.
| Reward scheme (Qwen3.5-9B, ALE-CM) | How the reward is built | S@1 | S@5 | GPU-hours per step | Source |
|---|---|---|---|---|---|
| Base model, no RL | none | 0.25 | 0.33 | – | Table 5 |
| Terminal only (sparse) | 1 if the verifier accepts, else 0 | 0.25 | 0.32 | 16.7 | Table 5 |
| PPRS, pooled normalization | ΔΦ per step plus terminal, advantages normalized per rollout | 0.28 | 0.35 | 10.9 | Table 5 |
| Direct additive reward | Φ(st) itself at each step, no credit passed back | 0.29 | 0.35 | 9.7 | Table 5 |
| PPRS (full recipe) | ΔΦ per step plus terminal, return-to-go with γ = 1, advantages normalized per turn | 0.29 | 0.40 | 9.0 | Table 5 |
Three details keep this from being just "add a partial-credit score." First, PPRS is not textbook potential-based shaping: classic shaping zeroes the potential at the end so the best policy never changes, while PPRS keeps the λΦ(sT) term and deliberately changes the objective. Second, the way the signal enters matters: in the 9B ablation, rewarding Φ directly at each step with no credit passed back to earlier actions led to more rollouts exhausting the context window before submitting, and produced longer trajectories. Third, the benefit grows with scale: at 27B the paper reports that shaped reward trains to completion, while terminal-only reward produces almost entirely zero-advantage groups and fails to complete a single optimizer step.
The limits are also in the paper. The 9B model gained only 0.04 in S@1 over sparse reward, and the headline Pass@5 match with frontier models is a statement about fully solved tasks, not average score. This is a reward-design result, so its value depends on having a latent world to read progress from; the paper builds that for 7 computing and math-science domains, not for open-ended work. For the evaluation side of the same idea, see why a score carries more information than pass or fail.
Goes deeper in: AI Agents → Evals & Diagnostics → Pass/Fail vs Score
Related explainers
- Long-Horizon-Terminal-Bench grades agent progress densely — the same partial-credit idea used to score agents instead of train them
- VPO — Vector-reward advantage vs GRPO scalar collapse — another fix for what a single scalar reward does to GRPO's advantages
- CoPD paper — Reinforcement Learning with Verifiable Rewards (RLVR) — the verifier-reward setting that PPRS builds on