What is the Causal Encoder-Decoder in DeepSeek-V4.1-Flash?

It is an architecture that gives prefill and decode **different active-parameter budgets inside one model**: about 8B active parameters during prefill and 16B per decode token, out of a 552B-parameter mixture-of-experts backbone. The name is an intuition rather than a formal claim: the input-heavy path plays the role of an encoder over the prompt, while the token-at-a-time path plays the decoder.

Why give prefill fewer parameters than decode?

Because prefill's parameter-dependent cost scales with **active parameters × prompt tokens**, while decode's scales with **active parameters × generated tokens**. One extra billion active parameters adds roughly 2 billion FLOPs per token — multiplied by a million tokens in prefill, and by however many tokens you actually generate in decode. Active parameters are cheapest in the phase where they are multiplied by the fewest tokens.

How is this different from prefill/decode disaggregation?

Disaggregation splits the two phases across **separate machines** so they stop interfering in the same batch, but both pools still execute the same active-parameter path. The Causal Encoder-Decoder splits the **active-parameter path**, so the two phases differ in how much of the model they use. Conceptually the two are complementary, because they divide different things — but the source does not describe a deployment that combines them.

What is SWA Bounded Replay?

It pairs sliding-window attention with a replay mechanism that limits how much of the KV cache must stay **persistent** on SSD or host memory. DeepSeek reports the result — a persistent footprint about **one eighth** of DeepSeek-V4-Flash — but the source does not describe how the replay itself works.

LLM· 2026-09-19

DeepSeek-V4.1-Flash runs prefill on 8B parameters and decode on 16B — Causal Encoder-Decoder — What does it mean?

TL;DR

What is it: DeepSeek-V4.1-Flash is a 552B-parameter multimodal mixture-of-experts model whose Causal Encoder-Decoder runs prefill through an 8B active-parameter path and decode through a larger 16B one — two active-parameter budgets inside one model.
Why it’s needed: Prefill and decode are different kinds of work — prefill is typically compute-bound, decode typically memory-bandwidth-bound — so a single active-parameter budget is a compromise for at least one of the two phases, and the longer the prompt, the more that compromise costs.
vs previous: Prefill/decode disaggregation already answers this by splitting the two phases across separate machines, but both pools still run the same active-parameter path; the Causal Encoder-Decoder splits the path itself, so the two phases differ in how many parameters they activate rather than in where they run.

Jargon

Prefill: The first phase of a request, where the model reads the whole prompt in one batched pass and fills the KV cache. Covered in LLM Internals → KV Cache.
Decode: The second phase, which emits one token per step and reads the whole KV cache back each time. It is typically limited by memory bandwidth rather than by arithmetic.
Active parameters: In a mixture-of-experts model, the subset of weights actually used for a given token. DeepSeek-V4.1-Flash has 552B total parameters but activates only 8B or 16B of them, depending on the phase.
Memory-bandwidth-bound: A workload whose speed is set by how fast bytes move from HBM, not by how fast the arithmetic units run. Decode is the standard example — see GPU & CUDA → Roofline.
HBM: High-bandwidth memory — the large, fast memory stacked next to the GPU die where the weights and KV cache live. Decode spends most of its time waiting on it. See GPU & CUDA → Memory Hierarchy.
KV cache: The stored keys and values for every token seen so far, so attention does not recompute them. Its size grows linearly with context length.
CSA2: Compressed Sparse Attention 2, the attention variant in this release. It reuses KV state across layers so one cached entry serves several layers instead of one.
SWA Bounded Replay: Sliding-window attention combined with a replay mechanism that limits how much of the KV cache has to stay persistent on SSD or host memory. DeepSeek reports the effect — a persistent footprint about one eighth of DeepSeek-V4-Flash — but the source does not describe how the replay itself works.

The news. A September 17, 2026 report introduces DeepSeek-V4.1-Flash, a 552B-parameter multimodal mixture-of-experts model with a one-million-token context window. It activates 16B parameters per decode token but only 8B during prefill — a split the report calls the Causal Encoder-Decoder. Alongside it, Compressed Sparse Attention 2 reuses KV state across layers, FP4 (four-bit floating point) compresses cached values, and SWA Bounded Replay cuts the persistent KV footprint to about one eighth of DeepSeek-V4-Flash. The global KV cache that must stay in HBM is 890 bytes per token, about a quarter of the previous model's. Read the report →

Picture the prep shift. A crate of vegetables comes in and the recipe gets applied to the whole crate at once — so if you add one step to that recipe, you have just added it a million times, once per item in the crate. Prefill is the prep shift: the model is applied to every prompt token in one batched pass, so one extra billion active parameters is arithmetic you pay for once per token in the prompt. That is why prefill typically sits on the compute-bound side of the roofline: batching the prompt's tokens together lets each weight be reused across many tokens, so the arithmetic units rather than the memory bus become the limit.

Now the service shift. Plates go out one at a time, and the recipe is applied once per plate. Decode is the service shift: one token per step, so the same extra billion parameters costs you once, not a million times. Decode's problem is a different one — between steps the GPU is mostly waiting on HBM to deliver weights and the growing KV cache, not waiting on arithmetic.

Most models hand both shifts the same recipe. A dense model has one parameter count by construction; a conventional mixture-of-experts routes to the same number of experts whichever phase it is in. The Causal Encoder-Decoder breaks that assumption: it gives the input-heavy path a smaller active-parameter path than the one-token-at-a-time path. So the cheapest place to spend active parameters is the phase where they are multiplied by the fewest tokens — which here is decode, the phase most people assume they have to economize on.

Prefill vs Decode on the Roofline

Same GPU, fundamentally different bottlenecks

Approach	What gets split	Do the phases activate different parameters?
One model, one pool	nothing — prefill and decode interleave in the same batch	No
Chunked prefill	the prompt, into slices that share a batch with decode	No
Prefill/decode disaggregation	the machines — one pool prefills, another decodes	No
Causal Encoder-Decoder	the parameters — a smaller active path for prefill	Yes

Put numbers on it. Hold one request fixed: a 1,000,000-token prompt and 1,000 generated tokens (the output length is illustrative — the report does not fix one). A forward pass costs roughly 2 FLOPs (floating-point operations) per active parameter per token, the standard rule of thumb. It counts only the parameter-dependent arithmetic — it leaves out attention over the context, cache traffic and everything else a request pays for, which at a million tokens is substantial — so treat these as one term of the bill, not the whole bill. Prefill touches 8B active parameters across all 1,000,000 prompt tokens: about 2 × 8 × 10⁹ × 10⁶ ≈ 1.6 × 10¹⁶ FLOPs. Decode touches 16B across 1,000 output tokens: about 2 × 16 × 10⁹ × 10³ ≈ 3.2 × 10¹³ FLOPs. On that term prefill does roughly 500× more arithmetic than decode, even though decode is the phase running the bigger path. Now give both phases a uniform 16B budget, changing nothing else: prefill's parameter-dependent work doubles to ≈ 3.2 × 10¹⁶ FLOPs while decode's stays exactly where it was. In this calculation, all of the extra parameter-dependent arithmetic lands on prefill, the phase that was already compute-bound. The KV side of the same request is where the other numbers live: at 890 bytes per token, a full 1M-token context holds about 890 MB of global KV in HBM before you serve a second user.

How much this wins depends on the shape of the request. Take a 200-token prompt with 1,000 output tokens (illustrative): shrinking the prefill path still cuts parameter-dependent prefill work, but that work is now a far smaller share of the request, so the saving shrinks with it. The arithmetic favors this design most when prompts are long — which is the regime a one-million-token model is built for. It also looks complementary to the machine-level split rather than a replacement for it, since the two divide different things: one divides the worker pools, the other divides the parameters. Disaggregation splits the machines; the Causal Encoder-Decoder splits the active-parameter path. The report stops there: it does not describe how the two paths share weights, how a request crosses between them, or how the KV cache is handed over.

Goes deeper in: LLM Serving → Prefill/Decode Disaggregation → Full Disaggregation

Related explainers

NVIDIA Dynamo — Encode-prefill-decode disaggregation — the same asymmetry answered by splitting worker pools instead of parameters
DeepSeek V4-Pro and V4-Flash — long-context cost — the previous generation's FLOPs and KV numbers, which this release compresses further
Latent Context LMs — encoder-decoder prompt compression — a different encoder-decoder split, cutting the number of positions rather than the parameter path

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based