What is an exponential race in LLM watermarking?

Each candidate token gets a secret lane length from the key and runs at a speed equal to its model probability. The first to finish is the sampled token, which has exactly its model probability of winning, and the detector can recompute the winner's lane length.

Why does the key not just decide every token?

If the key fully determines each token, the same prompt always gives the same answer. A watermark must leave some randomness to the sampler, so schemes differ in how much entropy the key removes.

What did TextSeal2 change from TextSeal?

TextSeal chose between two keys with a coin flip and the detector averaged both scores, reducing signal to noise by about √2. TextSeal2 merges the keys into one virtual key with a Gaussian copula, so every token scores against one key.

How is the copula strength chosen?

The paper reports that a fixed coupling strength works: ρ near 0.64, 0.81 or 0.93 removes about 30%, 50% or 70% of a token's entropy.

LLM· 2026-11-01

TextSeal2 watermarks LLM text through speculative decoding — Exponential-race sampling — What does it mean?

Jargon

Distortion-free watermark: A watermark whose next-token distribution, averaged over all possible secret keys, equals the model's own distribution.
Exponential race: A sampling method. Each token gets a lane length drawn from the key, runs at a speed equal to its probability, and the first to finish is the sample. Token v wins with a probability equal to its model probability.
Gumbel-max trick: The same idea written with Gumbel noise instead of exponential lane lengths. Aaronson and Kirchner (2023) applied it to keyed pseudo-random values to watermark text.
Gaussian copula: A way to blend two random values so that each stays uniformly distributed on its own while the pair has a chosen correlation, here called the coupling strength.
Virtual key: In TextSeal2, the single key that results from merging two or more keys' Gaussian values, so the detector scores one key on every token.
Self-BLEU: A measure of how similar repeated generations are to each other. Lower means more diverse output.
Temperature and sampling: The knobs that control how random token choice is. Top-K and Top-P sampling

The news. On 29 October 2026, Meta researchers posted TextSeal2, which the paper describes as a distortion-free watermark with copula key mixing and memoryless races. It is evaluated on three LLMs. Read the paper →

Normal sampling draws a token from the model's probabilities with a random number generator. A watermark replaces that generator with a keyed one. Picture the footrace: every candidate token runs in its own lane, and the lane lengths come from the secret key and the preceding words. The model's probability sets each runner's speed, so the runner that finishes first is a sample, and token v wins with exactly its own probability. That is why the scheme is called distortion-free.

Detection reads the same race backwards. The detector knows the key and the text, so it recomputes the keyed value behind each chosen token and scores how extreme it is. Chosen tokens tend to have more extreme keyed values than chance, and the sum of per-token scores, tested against its no-watermark distribution, separates watermarked text from human text. The same property raises the central problem. If the key fully determines every lane, detection is strongest but the same prompt gives the same answer every time. If the key is ignored, output varies freely and carries no signal. The paper puts it as the key removing a fraction of each token's entropy.

A worked example (illustrative numbers). Take three candidates with model probabilities mat 0.5, floor 0.3, rug 0.2 and key lane lengths 0.9, 0.2 and 0.6. Each finishing time is lane length divided by probability: mat 1.80, floor 0.67, rug 3.00. Floor finishes first at time 0.67, so floor is the sampled token. The other lanes have covered time times probability: mat 0.33 and rug 0.13. Their leftovers are 0.57 and 0.47. By memorylessness, those leftovers behave like new random lane lengths, which is what lets a second race reuse them without bias.

Scheme	How it trades diversity for signal	Detected (200 tokens)	Extra work per token
SynthID-Text	Tournament among sampled candidates	89 ± 3%	Grows with layers times candidates
TextSeal	Coin flip between two keys	81 ± 1%	None
textGrain	Optimal-transport coupling each step	89 ± 4%	Sinkhorn iterations, 1 to 3 ms per token
AutoMark copula	Single key blended with fresh noise	91 ± 3%	None
TextSeal2	Two keys merged, then copula	91 ± 2%	None

TextSeal2 keeps TextSeal's two keys, which support tampering and public detection, but drops the coin flip that costs signal. The coin flip fails because the detector does not know which key made each token and must average both, which reduces signal to noise by √2. The table above is the paper's comparison without speculative decoding on Qwen3.5-9B at fixed diversity. TextSeal2 merges the two keys' Gaussian values into one virtual key, then applies AutoMark's copula so fresh noise supplies the diversity. Any number of keys can be merged this way without diluting detection.

The coupling strength ρ sets how much entropy the key removes. The paper reports that ρ ≈ 0.64, 0.81 and 0.93 remove about 30%, 50% and 70% of a token's entropy across many distribution shapes, so one fixed ρ works without per-token tuning. Under speculative decoding, the same race structure extends to the replacement token, which is covered in the sibling explainer below.

Goes deeper in: LLM Internals → Text Generation → Top-K and Top-P Sampling

Related explainers

TextSeal2 and keyed replacement tokens — how one extra race keeps the mark when a draft is rejected

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based