What is IndexCache in Hy4 Preview?

IndexCache is the part of Hy4's attention that stores the sparse-attention index and reuses it across layers. Hy4 uses Gated DeepSeek Sparse Attention, where a cheap indexer picks the handful of cached tokens each query will actually attend to; instead of each attention layer rebuilding that selection, IndexCache stores it and lets later layers read it. Tencent names the mechanism and its purpose but does not publish how many layers share one index.

Why does reusing the sparse index across layers save anything?

Because building the index means scoring the whole cached past. At a 1M-token context that is a million scores per query per layer, and a 78-layer model repeats it 78 times for a single decoded token. Reusing an index removes that repetition for every layer that shares it, and Tencent does not publish how many do. It is a compute saving, not a memory saving — the KV cache still holds every token the index can point at.

How does IndexCache relate to DeepSeek Sparse Attention?

DeepSeek Sparse Attention is the selection mechanism: a lightning indexer scores cached tokens and each query attends to the top-k. IndexCache does not change what gets selected, only how often the selection is computed. The tradeoff is that a later layer inherits an earlier layer's choice rather than making its own, which is an approximation Tencent does not quantify in the model card.

LLM· 2026-09-07

Tencent open-sources Hy4 Preview at 770B — IndexCache cross-layer index reuse — What does it mean?

TL;DR

What is it: Hy4 Preview — Tencent's Apache-2.0 open-weight model with 770B total parameters, 49B active per token, 78 layers and a 1M-token context — runs Gated DeepSeek Sparse Attention with IndexCache, a mechanism that saves the sparse-attention index and reuses it across layers.
Why it’s needed: Sparse attention only pays off if deciding what to attend to is cheaper than attending to everything, and that decision — the index — is normally rebuilt inside every attention layer, so its cost scales with the depth of the model as well as with the length of the context.
vs previous: Where a sparse-attention scheme conventionally recomputes its index in each layer, IndexCache computes it once and lets later layers read the stored selection, trading some per-layer precision for one less repeated pass over a million cached tokens.

Jargon

Sparse attention: Attention where each query scores only a selected subset of the cached past instead of all of it. It is what makes a 1M-token context affordable, because dense attention's cost grows with the square of the sequence length.
Sparse index: The short list a sparse-attention layer produces: which cached tokens (or which blocks of them) this query is allowed to attend to. Everything not on the list is skipped.
Indexer: The cheap scoring pass that builds the sparse index. DeepSeek's version is called a lightning indexer; it scores every cached position quickly so the expensive attention math runs on only a few of them.
Top-k selection: Keeping the k highest-scoring candidates and dropping the rest. Here, k is how many cached tokens survive into the attention computation.
DSA (DeepSeek Sparse Attention): The sparse-attention design Hy4 builds on, which selects individual tokens rather than contiguous blocks. Full explainer
MoE (Mixture-of-Experts): A layer that holds many feed-forward sub-networks and routes each token to only a few of them. Hy4 holds 770B parameters this way but runs only 49B per token.
MTP (multi-token prediction): A small extra layer that proposes several next tokens at once, used as the draft model in speculative decoding. How draft-and-verify works

The news. On August 29, 2026, Tencent published Hy4 Preview on Hugging Face under Apache-2.0: a 770B-parameter Mixture-of-Experts model that activates 49B parameters per token across 78 layers, supports a 1M-token context, and ships in BF16 and FP8 with deployment recipes for vLLM and SGLang. Its attention is Gated DeepSeek Sparse Attention with IndexCache to reuse sparse indices across layers; a native MTP layer (10B total, 0.7B activated) supplies draft tokens for speculative decoding. Read the model card →

Picture the ticket rail above the pass. A runner walks out to table twelve, writes down what the table actually wants, and spikes that one ticket where every station can see it. The grill reads it. The sauté station reads it. Nobody walks back into the dining room to ask again, because the expensive part was the trip, not the reading.

Sparse attention has the same shape. A model carrying a million tokens of context cannot have every query score every cached token — that is dense attention, and its cost grows with the square of the sequence length. So a sparse scheme sends a cheap indexer out first: it scores the cached past, keeps the top handful, and hands the attention math a short list. Deciding which cached tokens a query is scored against is the whole game; the arithmetic that follows is the easy part.

The catch is that a transformer is not one attention layer. Hy4 Preview has 78 of them, and a sparse scheme conventionally sends a fresh runner from every single one. IndexCache spikes the ticket on the rail instead: the index is computed, stored, and read by later layers rather than rebuilt by each. Tencent's model card names the mechanism and its purpose — reusing sparse indices across layers — but does not publish how many layers share one index, so treat the sharing pattern itself as undocumented.

Reuse is only free if the layers actually want the same tokens. A query vector at layer 2 is not the query vector at layer 60 — every layer has rotated and reshaped its representation on the way up, so in principle each one would pick a slightly different top-k. Sharing an index means a later layer attends to an earlier layer's choice rather than its own. That is a genuine approximation, and it is the price IndexCache pays for the trip it skips.

The bet behind it is that sparse selections are largely shared between nearby layers — that which cached tokens matter for a given query is more a property of the content than of the layer doing the asking. Tencent publishes no evidence for that bet in the model card, and no ablation against a per-layer index. What is on the record is that the mechanism shipped inside a released open-weight model, which is weaker evidence than a measured accuracy delta and should be read that way.

Scheme	What each query attends to	How often the index is built
Dense (full) attention	every earlier token — there is no index	not applicable
DeepSeek Sparse Attention	the individual tokens a lightning indexer scores highest	conventionally, inside each attention layer
MiniMax Sparse Attention	whole KV blocks rather than single tokens	conventionally, inside each attention layer
Lookahead Sparse Attention	the cached chunks a neural memory indexer predicts will be needed	conventionally, inside each attention layer
Hy4: Gated DSA + IndexCache	the same token-level selection as DSA	once, then reused across layers — sharing pattern not published

Where the saving actually lands

Hold three things fixed: a 1M-token context, the 78 layers Tencent documents, and one token being decoded. An indexer that runs per layer has to score all one million cached positions to pick its top-k, so that single decoded token costs 78 × 1M = 78 million index scores before any attention arithmetic happens at all. Build the index once and let the other layers read it, and the same token costs 1 million — a 78× cut in indexer work. (Illustrative: Tencent does not publish how many layers share one index. If it refreshes every six layers the factor is 13×, not 78×.)

Two things keep that number honest. The indexer is deliberately cheap — a few operations per cached position, against a full attention head's worth of work for each token it selects — so this is not a 78× speedup of the model. And the saving is a compute saving, not a memory one: the KV cache still holds all million tokens, because a reused index still points at them. What IndexCache removes is a fixed overhead that sparse attention adds back on top of the cache, and that overhead is the thing that grows with depth. The deeper the model, the more of the sparse-attention tax IndexCache refunds.

Goes deeper in: LLM Internals → Attention → Computing Attention Scores

Related explainers

Tencent open-sources Hy4 Preview at 770B — Identity Hyper-Connections — the other architecture change in the same model: four residual streams between layers instead of one.
MiniMax M3 — MiniMax Sparse Attention (MSA) — the block-level alternative to the token-level selection Hy4 caches.

Continue in trackLLM Internals — Attention & Computing Attention Scores

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based