What is the denoiser, score, and bridge predictor equivalence?

It is the central result of a 2026 discrete-diffusion theory paper (arXiv:2607.05381): the three common ways to parameterize (train) a diffusion language model — predicting the clean token (denoiser), predicting probability ratios between neighboring states (score), or predicting what fills the gap (bridge/cavity) — are provably equivalent coordinate systems for the same underlying object, the reverse jump rate. There are closed-form formulas to convert any one into the others.

Why does it matter which parameterization a diffusion model uses?

Because the equivalence shows it largely does not at the optimum. Since the three recipes describe the same reverse process, the choice becomes one of numerical convenience and training stability rather than modelling power. The paper also shows the denoiser and bridge parameterizations coincide under masked noise but diverge under uniform noise, so the noise process — not the recipe — is the choice that actually changes the model.

How does this relate to autoregressive text generation?

Autoregressive models generate one token at a time, left to right, each conditioned on the tokens before it. Diffusion language models instead start from noise and refine a whole block of tokens in parallel, learning the reverse jump rate this paper analyzes. The equivalence result is about how you train that reverse process; it does not apply to the strictly sequential autoregressive decoder, which predicts a next-token distribution rather than a reverse jump rate.

Discrete diffusion theory unifies denoisers, scores & bridge predictors — One object, three coordinates

TL;DR

What is it: A discrete-diffusion theory paper shows that the three ways people train a diffusion language model — the denoiser, score, and bridge parameterizations — are all the same underlying object (the reverse jump rate) written in three different coordinate systems.
Why it’s needed: Diffusion is a leading non-autoregressive way to generate text, and the field has been split over which of these training recipes is "right." If they are provably equivalent, then at the optimum the choice is largely about numerical convenience, not modelling power — which frees engineers to pick whichever is easiest to train and serve.
vs previous: Earlier work treated recipes like MDM, SEDD, UDM, and GIDD as competing objectives you had to choose between. This paper shows they are inter-convertible coordinates of one optimizer, with closed-form formulas between them — so they are not rivals, they are the same theory in different clothes.

Jargon

Discrete diffusion model: A text generator that starts from noise and refines a whole block of tokens at once, over a few steps, instead of writing left to right. "Discrete" means the noise acts on tokens (a vocabulary), not on continuous pixels. See how the usual left-to-right decoder works for the contrast.
Reverse jump rate: The single quantity a diffusion model must learn: for a corrupted slot, the rate at which the true reverse process would jump that token back toward its clean value. Everything the model predicts is a way of describing this one object.
Denoiser (x0-prediction): One parameterization: the model directly predicts the clean token that was corrupted — "the word here was river." The most intuitive of the three.
Concrete score: Another parameterization: instead of the clean token, the model predicts the ratios of probability between neighboring states — the discrete analog of the score (the gradient of log-density) used in continuous diffusion.
Bridge / cavity predictor: A third parameterization that predicts the token through a bridge (or "cavity") representation — roughly, what should fill this position given everything else. For masked noise it coincides with the denoiser; for uniform noise it does not.
ELBO: The evidence lower bound — the quantity the model is trained to maximize (equivalently, the negative ELBO is the loss it minimizes). The paper gives it a precise meaning as a distance to the perfect reverse process.
Masked vs uniform diffusion: Two kinds of forward noise. Masked replaces tokens with a [MASK] symbol; uniform replaces them with a random token. The paper shows the parameterizations coincide under masking but split apart under uniform noise.

The news. On July 6, 2026, a research paper (arXiv:2607.05381) asked a deceptively simple question: what does a discrete diffusion model actually learn? Using a continuous-time Markov-chain view of how a sequence gets noised, it proves an "Oracle Distance" theorem linking the training loss to the distance from a perfect reverse process, then shows that the denoiser, score, and bridge parameterizations are equivalent coordinates for the same optimizer, with closed-form conversions between them. Prior recipes — MDM, UDM, SEDD, and GIDD — fall out as special cases. Read the paper →

Picture standing at the mouth of a parking garage, needing to tell someone how to reach one specific car. You could give the exact address — "spot B-12." You could give a compass bearing — "head northeast and keep bearing right." Or you could name a landmark — "it's under the big pipe." Three completely different-sounding instructions, one destination. And if you know any one of them precisely, you can work out the other two. That is the whole result of this paper, transplanted onto diffusion language models.

Start with the destination. Unlike a normal chatbot that writes one token at a time, strictly in order, a diffusion language model begins with a sequence of noise and cleans it up over a few passes, editing every position in parallel with attention that can look both left and right. To do that, the model needs to know, for each corrupted slot, which clean token it should move back toward — formally, the reverse jump rate. This single quantity is the one destination; it is the only thing the model is really trying to learn.

Here is where the three "ways to give directions" come in. The denoiser recipe predicts the clean token outright — the exact address. The score recipe predicts the ratios of probability between neighboring states — a compass bearing that says which way to shift the odds. The bridge recipe predicts what should fill the gap given the rest — the landmark. The paper proves these three are equivalent coordinate systems for the same reverse jump rate, with exact closed-form formulas to convert between them — so, at the optimum, a debate over "which objective is correct" is really a debate over which set of directions is most convenient to write down.

Coordinate	What it predicts	Directions analogy	Recovers
`denoiser`	the clean token directly	the exact address	MDM
`score`	ratios between neighboring states	the compass bearing	SEDD
`bridge` / cavity	what fills the gap given the rest	the landmark	UDM, GIDD

Coordinate equivalence and the recovered prior methods are from the paper (arXiv:2607.05381); the analogies are illustrative.

To see the conversion concretely, picture a single masked slot whose true word is river. The denoiser coordinate reports a probability over the vocabulary — say 0.6 on river, 0.25 on liver, the rest spread thin (illustrative). The score coordinate never stores those probabilities directly; it stores the ratio between neighboring words — river over liver = 0.6 / 0.25 = 2.4 (illustrative). The bridge coordinate keeps a third view. The theorem works at the level of the whole coordinate systems, not a single scalar: it gives fixed, closed-form formulas that rewrite any one of the three parameterizations as another, so training in one and reading out in another needs no retraining and no extra data. For a masked slot the denoiser and bridge coordinates even collapse to the same object; only under uniform noise do they split apart — which is why the choice of noise, not the choice of recipe, is what actually changes the model.

And the loss has a clean meaning too. The paper's Oracle Distance theorem shows the negative ELBO — the thing you minimize during training — splits into a constant (the data's own entropy) plus the distance between your model's denoising path and the path a perfect reverse process would take. The constant does not depend on the model, so the only part you can improve is exactly that path distance — driving the loss down is literally closing the gap to the ideal denoiser, a far more satisfying story than "minimize this bound because the algebra works out."

Goes deeper in: LLM Internals → Text Generation → One Token at a Time

Related explainers

DiffusionGemma — Parallel block decoding — the decoding side of the same idea: refining a whole block of tokens at once instead of one at a time.
Parallel speculative decoding for diffusion LLMs — how diffusion LMs are made fast enough to serve.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based