AI Explained
Plain explanations of trending AI concepts, with live visualizations.
TIM paper — Training-Inference Mismatch in RL — What does it mean?
Zhong et al. introduce a controlled diagnostic, VeXact, that isolates rollout/policy numerical drift from every other RL instability — and show that drift alone, on the same nominal weights, is enough to collapse training.
SP-KV paper — Utility predictor for the KV cache — What does it mean?
Meta FAIR's SP-KV learns to write only high-value KV pairs to cache — 3-10× smaller footprint with little-to-no validation-loss or task-performance drop.
Quantization-conditioned attack paper — Outlier injection across AWQ/GPTQ/GGUF — What does it mean?
A quantization-conditioned attack hides one outlier in a weight block — AWQ / GPTQ / GGUF I-quants then stretch their per-block scale to fit it, collapsing most other weights toward zero. FP16 benign, INT4 malicious.
PreFT applies LoRA only to prefill — Prefill-only LoRA adapters — What does it mean?
Stanford's PreFT runs the LoRA adapter during prefill, then drops it before decode begins — the adapter's behavioural signal lives inside the KV cache it shaped, so decode runs the bare base model and serves 1.9× the requests on 512 concurrent adapters.
TFGN paper — Subspace-preserving updates for continual pre-training — What does it mean?
TFGN continually pre-trains an 8B LLM without replay buffers or task IDs by structuring each update to live in a subspace orthogonal to prior-domain knowledge — backward transfer −0.007, JS perplexity −26.8% from Python-only training.
HuggingFace blog — Async continuous batching — What does it mean?
Sync continuous batching stalls the GPU while Python composes the next batch — overlapping CPU prep with GPU compute lifts GPU-active time from 76% to 99% in HuggingFace's report (~22% speedup).
Compute Where It Counts — Per-token compute controller — What does it mean?
An ICML'26 paper bolts a lightweight policy network onto a frozen LLM that picks a per-token efficiency action — attention sparsity, MLP pruning, or activation bit-width — so easy tokens get cheap actions and hard tokens get full compute.
SOP paper — Hardware-aware per-layer PTQ at FP6 — What does it mean?
SOP picks a different codebook per layer using activation weights — and at FP6, that beats vanilla FP8 reconstruction error using 1.5 fewer bits per weight.
PPOW paper — window-level RL for speculative drafters — What does it mean?
PPOW trains speculative-decoding drafters with WINDOW-level RL — three rewards adapt window size to KL divergence, lifting acceptance to 6.29–6.52 and end-to-end speedup to 3.4–4.4×.
vLLM v0.20 — TurboQuant 2-bit KV cache — What does it mean?
vLLM v0.20 ships TurboQuant — a 2-bit KV cache with per-block scales, cutting KV memory ~4× without crushing outliers.
vLLM v0.20 — FlashAttention 4 packing — What does it mean?
vLLM v0.20 ships FlashAttention 4 with packed variable-length attention — one fused kernel for a whole batch, no padding waste.
NVIDIA Nemotron 3 Nano Omni — 30B-A3B multimodal MoE — What does it mean?
Nemotron 3 routes text, image, audio, and video through ONE shared expert pool — 30B in HBM, ~3B active per token, ~9× throughput.