AI Explained
Plain explanations of trending AI concepts, with live visualizations.
FlashOffloading offloads all KV cache — rolling prefill buffers plus a hot decode cache — What does it mean?
FlashOffloading keeps the master KV cache in host DRAM and overlaps transfers with compute, so each GPU serves more long requests.
SpecCTMC — Exact speculative sampling for masked diffusion LMs — What does it mean?
SpecCTMC drafts several diffusion-LM unmasking steps, verifies them in one batch, and corrects the first miss, staying exact.
AMD ROCm InferaSim — Sparse GPU anchors and transfer laws — What does it mean?
AMD's InferaSim predicts serving configs from one anchor per kernel set; 11 anchors took 9.1 GPU-hours versus 204 to measure.
HyFlash lifts on-device speculative decoding 16.2% — Prefix-memory drafting — What does it mean?
A drafter with a local window plus fixed-size recurrent memory keeps long context in 5 MiB.
Measured Coupling paper — Parallel diffusion decoding by measuring token coupling — What does it mean?
Diffusion decoders commit tokens together; measured coupling tests which pairs are safe instead of guessing.
DictLLM compresses LLaMA-7B 5.1× — Shared dictionary plus sparse coefficients — What does it mean?
DictLLM stores one shared dictionary per layer group plus sparse coefficients, shrinking LLaMA-7B 5.1× without retraining.
Adversarial speculative reasoning paper — rejecting drafts on purpose — What does it mean?
Train the draft model to be rejected at an agent's decision points, so the target resamples them at slot 0, where numeric noise is lowest.
Restore MoE refusal safety with full-expert response starts — SafeStart warm start — What does it mean?
SafeStart keeps most of MoE expert-reduction's speedup by using every expert for the prompt and first 8 tokens, largely restoring refusals.
D3D diffusion drafter — Asynchronous speculative decoding — What does it mean?
A diffusion drafter without target hidden states drafts during verification, reaching 5.00× and 5.71× average speedups.
vLLM vs SGLang benchmark gap — Chunked-prefill confound — What does it mean?
Matching one chunked-prefill setting removed 78% of the vLLM-SGLang capacity gap, so default benchmarks overstate what the cache does.
Frontier LLMs edit weights directly — Weight-space programming — What does it mean?
An LLM edited 131,072 weights of a small model, with no training, and lifted its MATH-500 score from 7.9% to 48.1%.
ARLoop — Attention residuals over recurrent depth — What does it mean?
ARLoop lets each looped layer attend over earlier depth states, and a short window of recent blocks is enough.