AI Explained
Plain explanations of trending AI concepts, with live visualizations.
TextSeal2 watermarks LLM text through speculative decoding — Keyed replacement tokens — What does it mean?
TextSeal2 keeps the watermark on tokens that replace rejected drafts, with no loss of acceptance.
DolphinRouting — Local-vs-remote prefill routing — What does it mean?
DolphinRouting learns one local-or-remote prefill bit per request cluster, cutting P90 TTFT 5.7–7.9% vs Dynamo.
Raincheck cuts first-turn TTFT up to 24.1% — Deferred multimodal prefill — What does it mean?
Raincheck prefills pruned multimodal tokens during decode, keeping context while cutting TTFT.
Rimuru unifies serving memory — Shared page pool for KV cache and MoE experts — What does it mean?
Rimuru lets KV cache and MoE expert weights share one GPU page pool, so the boundary between them moves with load.
Speculative decoding can raise energy at large batches — Energy per accepted token — What does it mean?
Speculation saves energy only on accepted tokens; the saving fades with batch and can flip to a loss.
Skip MoE channels for up to 2.69× faster MoE blocks — top-k channel selection — What does it mean?
Top-k channel selection fixes the MoE read budget per token, with no calibration data.
Constrained decoding is not the culprit — Format, validity, and reasoning constraints — What does it mean?
Most JSON-mode accuracy loss comes from mismatched prompts, not from the decoder mask.
Prefetch-steered routing — Logical vs physical expert cache — What does it mean?
Route on a deterministic logical cache so prefetch timing never changes MoE outputs.
NgramRL paper — Adaptive fan-out prefetching — What does it mean?
NgramRL pre-builds next-step drafts for the likeliest verification outcomes so the GPU does not wait on the CPU.
NgramRL paper — Per-prompt rollout n-gram drafting — What does it mean?
NgramRL drafts RL rollout tokens from a per-prompt n-gram cache of the policy's own recent rollouts, then verifies exactly.
Pass soft posterior messages before committing low-bit weights — Message-Passing Quantization — What does it mean?
MPQ keeps a probability over codewords for every weight block and rounds only at the end, instead of committing block by block.
HyScale sparse-attention training — Selection without a score table — What does it mean?
HyScale finds sparse attention's top-k keys without a score table and replays them in recomputation, training 2.07× faster at 1M tokens.