Pass soft posterior messages before committing low-bit weights — Message-Passing Quantization — What does it mean?

The news. On October 23, 2026, Yanshuo Chen, Chenxi Liu, Reza Shirkavand and Heng Huang (University of Maryland) posted the preprint MPQ: A Message-Passing View of Post-Training Quantization (arXiv 2610.32127). It casts row-wise post-training quantization as posterior inference over discrete codebooks, and reports better perplexity and zero-shot accuracy than strong hard-decision baselines on dense Llama, Qwen and Mistral models, at 2-bit lattice and 3- and 4-bit scalar settings. Read the paper →

Picture one row of a weight matrix as a crossword. Each block of a few weights is a slot, and each slot has a short list of codewords the low-bit format is allowed to write there. The goal, as in the quantization process, is to choose words so the layer's output on a small calibration set barely moves. A greedy solver such as GPTQ fills this crossword in pen, slot by slot: it writes the word that fits best right now, then lets the unfilled slots soak up whatever mismatch is left.

MPQ fills it in pencil first. Every slot keeps all its plausible words with a confidence beside each, and crossing slots keep telling each other which letters are still possible. Only when the pencil marks settle does it ink one word per slot. The paper's framing makes the relationship exact: existing solvers are the zero-temperature limit of the same inference problem, the case where only the single top codeword per block is kept.

The number line above shows why a block has several near-tied codewords: low-bit grids are coarse, so two neighbouring values often fit almost equally well, which is the moment a hard rounding rule picks one and forgets the other.

MPQ runs in two stages. Stage one stays soft. For a fixed number of iterations it alternates between a denoiser, which gives each block a probability over its codewords, and a linear module, which solves a smooth least-squares problem with the calibration statistics and so spreads information between blocks. Stage two hardens. It takes the most probable codeword per block as a starting point, then runs exact block coordinate descent: re-check each block against its neighbours, switch if the error drops, and repeat for a few sweeps. Both stages are offline; nothing about inference changes.

Worked example (illustrative). Hold three things fixed: two correlated blocks, two candidate codewords A and B for block 1, and a squared-error score where lower is better. Alone, A leaves an error of 0.10 and B leaves 0.11. Because block 2 is correlated with block 1, the best block-2 word leaves 0.30 after A but only 0.18 after B. A greedy solver sees that 0.10 beats 0.11, commits A, and finishes at 0.10 + 0.30 = 0.40. A solver that carries "A 52%, B 48%" forward until block 2 has weighed in finishes at 0.11 + 0.18 = 0.29. The 0.01 edge at step one cost 0.12 at step two, and a rule that commits immediately cannot see that.

The paper's cleanest comparison fixes the model and the compute: Llama-3-8B, WikiText-2 perplexity (lower is better), and the 2-bit E8P12 lattice codebook. MPQ is run with 8 or 16 VAMP iterations plus 3 CD sweeps, against LDLQ with 3 or 6 CD sweeps, at roughly the same solver time.

Two points from the paper keep this honest. MPQ costs more offline time than plain greedy rounding: at 3 bits it matches GPTQ+CD3 at all four Llama sizes from 1B to 13B but stays several times slower than GPTQ alone, and at 2-bit E8P12 it stays within 3× of LDLQ. That cost is paid once, and the output is stored in GPTQ-compatible scalar or QuIP# E8P12 formats, so with matched grouping and packing the corresponding inference kernels run it.

The soft stage matters differently at different bit widths. At 3-bit scalar, VAMP alone already saturates quality and extra CD sweeps barely help. At 2-bit lattice, VAMP-only runs sit above 16 perplexity and every extra CD sweep pulls the number down, so message passing mostly picks the right basin and CD makes the precise commitment. These are the authors' own results on dense models, on one solver-timing setup (an RTX 5090).

Goes deeper in: LLM Internals → Quantization → Modern Methods

Related explainers

• E8-QAT: 2-bit training with an implicit E8 lattice codebook — the same 8-weight lattice codewords, reached by training instead of by post-training inference
• Why PTQ works: counteracting quantization error — why layer-by-layer rounding errors can cancel, the effect a hard-commit solver leans on