MoEZip is a KV-cache compression method for sparse Mixture-of-Experts language models. It keeps the cached tokens that attention finds important and also the tokens that keep the router's expert choices stable, within the same total cache size.

Why does KV eviction affect expert routing?

The router reads a hidden state that is produced by attention over the KV cache. Removing cached entries changes that hidden state, which can change the router's scores and therefore which experts are selected. The paper measures this as an expert-flip rate that rises as compression becomes more aggressive.

How does MoEZip decide which tokens to protect?

It computes a Fisher-trace score of the router's distribution at the instruction tokens, combines it with instruction-to-context attention, and reserves the top-scoring positions as anchors. A base scorer such as KVZip fills the rest of the budget.

How much does it improve results?

At a 10% KV retention ratio the paper reports average gains of 24 points on LongBench and 20 points on RULER over the prior state-of-the-art baseline, KVZip. These figures come from the authors' own evaluation on the models they tested.

LLM· 2026-10-25

MoEZip keeps expert routing stable at 10% KV retention — Routing-aware KV eviction — What does it mean?

Jargon

KV cache: The stored key and value vectors of every earlier token, so the model does not recompute them at each step. Its size grows linearly with context length. How it works
KV eviction: Deleting some cached tokens so the cache fits a memory budget. The question is always which tokens to delete without hurting the answer.
Mixture-of-Experts (MoE): A model whose feed-forward block is split into many experts, and only a few run for each token. The dense feed-forward block it replaces
Router: A small linear layer that reads a token's hidden state and scores every expert. The top-scoring experts are the ones that run for that token.
Keep ratio (retention ratio): The fraction of the KV cache that survives eviction. At a keep ratio of 10%, a 32,000-token context keeps 3,200 context positions per layer and head.
Fisher trace: A number computed from the router's output probabilities that measures how strongly the routing distribution changes when the router's input logits are nudged. A high value means a fragile routing decision.
Anchor: In MoEZip, a context position whose KV entry is reserved because evicting it would likely flip the router's choices. The remaining budget is filled by an ordinary scorer such as KVZip.

The news. On October 21, 2026, researchers from Korea University and SK Telecom posted MoEZip, a KV-cache compression method built for sparse Mixture-of-Experts models. At a 10% KV retention ratio the paper reports average gains of 24 points on LongBench and 20 points on RULER over the best prior baseline, KVZip. Read the paper →

Picture the dispatcher at a help desk. Every caller is sent to one of several specialist desks, and the dispatcher decides by reading the case file in front of them. That is an MoE layer: the router reads the model's current state, which was built from the KV cache through attention, and picks the few experts that will run. Now the building runs out of shelf space, so someone shreds pages from the case file. A dense model is like a single generalist desk: a thinner file means a slightly worse answer. An MoE model has a second failure. A thinner file can change which desk the caller is sent to, and the answer can change with it.

The authors measured this directly with an expert-flip rate, the share of tokens whose selected experts differ between a full-cache run and a compressed-cache run. On Qwen3-30B-A3B they found that answer quality fell sharply as eviction became more aggressive, while the flip rate rose in step. The dense Qwen2.5-7B-Instruct degraded far less under the same KVZip compression. Existing eviction methods rank tokens by how much attention they receive, so they have no way to see this second failure.

MoEZip adds a second ranking, in three moves.

Measure how fragile the router is. For each instruction token at each MoE layer, compute the Fisher trace of the router's distribution. This is cheap to compute and works for both softmax routers (Qwen3) and sigmoid routers (GLM-4.7-Flash).
Carry that fragility onto the context. The answer tokens do not exist yet when eviction runs, so MoEZip uses the instruction tokens as a stand-in. Each context position gets a score that combines how strongly the instruction attends to it with how fragile the routers are at those instruction tokens. The top-scoring positions become anchors.
Split the budget. Anchors are reserved first. A base scorer such as KVZip or SnapKV fills the rest. A sample whose routers look fragile gets more anchors, and a stable sample defaults mostly to the base scorer.

The cache size and the decoding procedure do not change; only which entries fill the cache changes. The paper reports the extra selection cost as about 0.88 seconds on Qasper.

Method	Full KV	10% KV, as is	10% KV, routing restored
KVZip-retained cache	53.7	19.5	29.7
MoEZip-retained cache	53.7	44.9	46.2

The table shows LongBench scores from the paper's controlled test on Qwen3-30B-A3B at a 0.1 keep ratio. In the last column, the retained cache is held fixed and only the instruction tokens' expert choices are overwritten with those of the full-cache run.

Here is the worked example, using the paper's own numbers. With the KVZip cache at 10%, the LongBench average falls from 53.7 to 19.5, a loss of 34.2 points. Restoring only the instruction tokens' expert choices, with every evicted entry still gone, lifts the score from 19.5 to 29.7. That is 10.2 points, about 30% of the entire loss, recovered by fixing routing alone. Back in the case-file picture, nobody gave the pages back; the dispatcher was only told which desk the full file would have chosen. The MoEZip cache loses 8.8 points (53.7 to 44.9), and the same routing correction recovers only 1.3 more. Its anchors had already protected most of the routing. Answer negative log-likelihood tells the same story: KVZip goes from 4.22 to 5.67 (4.97 once routing is restored), and MoEZip only to 4.79 (4.52 once routing is restored). At a 32,000-token context, 10% means 3,200 retained positions per layer and head in both cases, so the gap is not a memory difference.

Three limits are worth keeping in mind. First, the Fisher trace is a local, direction-agnostic proxy for sensitivity, not a measurement of the exact shift that eviction causes. Second, the paper itself notes that correcting instruction routing does not recover every effect of eviction, so routing is one cause of the loss and not the only one. Third, the default MoEZip reads the downstream instruction, so a compressed cache is tied to that query; the paper's query-agnostic variant, MoEZip-Recon, trades some of that conditioning for a cache that later requests can reuse. The reported tests cover Qwen3-30B-A3B, GLM-4.7-Flash and, in an appendix, Qwen3-235B-A22B.

Goes deeper in: LLM Internals → KV Cache → Memory Cost

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based