The news. On July 14, 2026, the Ring-Zero paper studied reinforcement learning with verifiable rewards at trillion-parameter scale, arguing that the many small-model "zero-RL" studies before it miss the dynamics that only show up when the model is huge. Its Ring-2.5-1T-Zero setup has 1 trillion parameters, is evaluated on 7 mathematical benchmarks, and is scored not just on final answers but on reasoning-trace quality — comprehensibility, reproducibility, and efficiency — judged by a separate framework. Its stabilized pipeline reports a two-phase learning pattern and emergent behaviors such as self-verification and parallel reasoning. Read the paper →
Picture a chess player locked in an empty hall with a scoreboard and no coach. Nobody shows them an opening book, nobody annotates a single game — the only signal that ever comes back is the final result, a win or a loss. That result is a verifiable reward: unfakeable, unarguable, and cheap to check. Learning a whole game from nothing but that verdict is exactly what zero-RL does to a language model — reward-only reinforcement learning applied straight to the raw pretrained model, with no supervised warm-up on worked solutions first. The "zero" is the missing coach.
Left to only win/loss, our player doesn't improve smoothly. They go through two phases. Early on they play wildly — trying strange openings all over the board, casting a wide net to discover what even works. That is the discovery phase: high variety, spreading bets across many lines. Only later, once a few lines keep winning, do they enter the sharpening phase — narrowing onto those winners and polishing them. The Ring-Zero paper reports that a language model trained with zero-RL shows this same two-phase shape — a broad discovery pass, then sharpening — and that at trillion scale a third thing appears on its own: the model starts verifying its own moves mid-game and running several lines of thought in parallel, behaviors nobody put there.
The catch is that a lone player replaying their own games at full intensity can spiral into bad habits, and the bigger the player, the harder the spiral. Ring-Zero's contribution is a stabilized pipeline that keeps trillion-scale self-play from destabilizing — three guardrails working together. Clipped importance sampling lets the model learn from rollouts its slightly older self produced, but clips how much any stale game can pull the update. Training-inference ratio correction fixes the fact that the code path which generates a rollout computes a little differently from the code path that trains on it — a gap that, as the model samples token by token, compounds into instability. And mixed-precision control keeps the low-precision arithmetic from drifting. Without these three controls the loop can destabilize as the model grows; with them, Ring-Zero holds it steady all the way to 1T.
Why bother proving this at a trillion parameters? Because the whole bet is about scale changing the verdict. Split one run into its two phases and watch the exploration budget. In the discovery phase a model spends graded attempts casting around; call it ~1,000 verifier checks (illustrative) to first surface a hard problem's solution and make it stick. The paper's headline is that a bigger model doesn't just brute-force more attempts: the same result surfaces in far fewer graded tries, and the ceiling it can reach climbs higher. In the authors' words, "Scaling to 1T parameters significantly enhances sample efficiency and performance ceilings" — the opposite of the worry that reward-only RL would get flakier as models grow. How much the model explores versus locks in is the same exploration-vs-exploitation dial that governs everyday sampling, now playing out across a whole training run.
| Recipe | Warm-up before RL | Reward signal | What Ring-Zero adds |
|---|---|---|---|
| SFT + RLHF | SFT on worked solutions | Learned reward model (human taste) | — |
| RLVR (post-SFT) | SFT on worked solutions | Deterministic verifier (correct / incorrect) | — |
| Zero-RL @ 1T [paper] | None — RL straight on the base model | Deterministic verifier (correct / incorrect) | Stabilizers + two-phase dynamics at trillion scale |
The honest caveats. The ~1,000 checks above is illustrative — the paper reports sample-efficiency gains and benchmark results, not a single headline accuracy number, and its evidence is on 7 math benchmarks, where "correct" is a clean yes-or-no a verifier can decide. Whether the same two-phase story holds on fuzzy, open-ended tasks is exactly the boundary RLVR itself runs into. But the deeper claim travels: once reward-only RL is stable enough to run at a trillion parameters, "skip the imitation stage and just let it learn from outcomes" stops being a small-model curiosity and becomes a real recipe for frontier reasoning models — with the model's own two-phase learning curve, not a human's worked examples, doing the teaching.
Goes deeper in: LLM Internals → Text Generation → Greedy vs Creative
Related explainers
- CoPD paper — Reinforcement Learning with Verifiable Rewards (RLVR) — the pass-or-fail verifier signal zero-RL runs on; CoPD applies it after an SFT stage, where Ring-Zero drops SFT entirely.
- TIM paper — Training-Inference Mismatch in RL — the generate-vs-update gap Ring-Zero's ratio correction is built to close.
- SIS — Selective Importance Sampling — another take on re-weighting off-policy rollouts, the same lever as Ring-Zero's clipped importance sampling.
- VibeThinker-3B — diversity-driven RL — the small-model counterpoint: at 3B, RL's tendency to collapse variety is fatal; Ring-Zero argues scale changes that verdict.