What is policy-adaptive skill injection?

It is a training-time rule: a written skill is added to the agent's prompt only when a cheap test says it helps the agent's current ability on that task. Otherwise the agent attempts the task with no skill. SkillScaffold uses it so skill guidance is neither assumed helpful nor tested with expensive new episodes.

How does SkillScaffold decide whether a skill helps?

It replays attempts the agent already made and scores them twice with the same model, once with the candidate skill in context and once without. A skill scores well if it makes recorded successful moves more likely and recorded failed moves less likely. The best candidate is injected only if its score is above zero.

Does the final agent still read skills at inference?

No. The paper removes the skill Generator and all skill text at deployment. The skills act as scaffolding during reinforcement learning, and the Executor is trained so useful behavior can carry over into its own weights; the paper reports analyses that support this.

How is this different from routing skills at inference time?

Skill routing, such as Gavel, picks a skill for each request at inference. SkillScaffold decides which skills to use while training and then discards them. The two answer different questions and can in principle be combined.

Agent· 2026-10-18

SkillScaffold injects skills only when they help — Policy-adaptive skill injection — What does it mean?

TL;DR

What is it: SkillScaffold (arXiv, 13 October 2026) is a training scheme for agents built on policy-adaptive skill injection: during reinforcement learning, a written skill is handed to the agent only when a cheap test, run against the agent as it is right now, says it will help.
Why it’s needed: In the authors' pilot, skills helped most where the agent was weakest and could hurt where it already succeeded, so guessing which skill helps by running more practice episodes is slow and noisy; the scheme replaces that with one replay of attempts the agent already made.
vs previous: Earlier skill-based RL either injects every generated skill without testing it or measures each skill by running fresh rollouts with and without it; SkillScaffold scores each skill by whether it makes the model find recorded successful attempts more probable and failed ones less probable, and abstains when no skill scores above zero.

Jargon

Agentic RL: Reinforcement learning where a language-model agent acts over many turns in an environment and is rewarded by a verifiable outcome, such as "the task was completed".
Rollout: One full run of the agent through a task, from the first step to the outcome. Rollouts are the expensive part of agentic RL because each one needs the environment.
Skill (skill card): A short piece of written procedure the agent reads before acting: when it applies, the steps, and how to check or recover. The curriculum covers skills under Skills as a Primitive.
Scout: This paper's name for a skill-free rollout. Scouts are collected first, labeled success or failure, and then reused as evidence.
Executor and Generator: Two roles played by one shared model. The Executor acts in the environment; the Generator writes candidate skills from the Scouts' evidence.
Credit assignment: Deciding which ingredient deserves credit for an outcome. Here the ingredient is a skill: did this skill make success more likely, or was it noise?
GRPO: Group Relative Policy Optimization. It samples a group of attempts at the same task and rewards each one relative to the group's average, so no separate value model is needed.
Skill internalization: The model absorbing a skill's behavior into its own weights, so the skill text is no longer needed at inference.

The news. On 13 October 2026, researchers from Zhejiang University and Kuaishou Technology posted "SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection". The claim is that an RL-trained agent learns better when generated skills are tested cheaply and injected selectively, and that the finished model then runs with no skills at all. Read the paper →

Stay with the coach. A student is preparing for an exam, and the coach writes hint cards. The naive coach puts a card on the desk for every practice problem. The paper's first finding is that this is a bad habit, because a hint card is not a free gift: it helps a student who is stuck and gets in the way of a student who already knows the move. In the authors' pilot, average skill gains were positive when only a minority of four skill-free probe attempts succeeded, but turned negative when most or all succeeded. Even among candidates for the same query, some helped and some hurt, so a plausible-sounding skill is not evidence of a useful one.

The obvious fix is to test each card: let the student redo the problem with and without it, many times, and compare. The paper measured how well that works and found it costly and unstable. Two independent estimates of the same skill's benefit disagreed on the sign 23% of the time with 1 extra rollout and still 16% of the time with 8. Raising the extra budget from 5 to 40 rollouts per query (8×) only moved the share of query groups with a distinguishable best candidate from 32% to 55%, leaving nearly half unresolved. Sparse success-or-fail rewards give each rollout very little signal, so the test needs many of them. Online RL has to decide now, which is why this matters for an agent's training loop and not only for evaluation.

Scoutskill-free attempts, labeled success or failure

GenerateGenerator drafts several candidate skills

Scorereplay the same Scouts with and without each skill

Gateinject the best skill only if its score is above zero

Trainenvironment reward updates the Executor, skill score updates the Generator

The five stages above are one training step. The cheap test is the third one. Instead of asking the student to redo the problem, the coach replays the student's recorded attempt and asks how plausible each move looks with the card on the desk versus without it. Technically, the same model scores the same recorded Scout tokens twice, once with the candidate skill in its context and once without, and compares the log-likelihoods (how probable the model finds those tokens). A skill is good if it makes the successful moves more likely and the failed moves less likely; the paper weights those two kinds of evidence equally, so a group with many failures and one success is not dominated by the failures. No new episode is run, which is how it avoids the rollout cost. The best-scoring candidate is injected into fresh Executor rollouts only if its score is positive. Otherwise the problem is attempted with no card, and the Generator learns from the score too, so its cards drift toward what this particular Executor can use. For the same trade-off seen at inference time, compare the Gavel explainer, which routes skills with no training loop at all, and the tool-loading strategy step.

Put numbers on it. Hold the candidate sets fixed: the paper reuses the same query-and-candidate sets from its pilot and compares four decision rules on them. Doing nothing succeeds on 61.8%. Injecting a random candidate gives 60.1%, which is worse than doing nothing. Always injecting the top-scored candidate gives 63.0%, only +1.2 points. Injecting the top-scored candidate only when its score is positive gives 67.6%. The gate is +5.8 points over doing nothing, and +4.6 of those 5.8 come from the abstention step alone. Ranking helps a little; knowing when to keep the card in the pocket does most of the work.

Decision rule	Success rate	Change vs no skill	Source
No skill (skill-free)	61.8%	baseline	arXiv §5.4
Random candidate	60.1%	−1.7 points	arXiv §5.4
Top-scored candidate, always injected	63.0%	+1.2 points	arXiv §5.4
Top-scored candidate, only if score above zero	67.6%	+5.8 points	arXiv §5.4

A tiny sign example shows the gate. Say a Scout group has some successes and some failures, and two candidate cards are scored on d, the average per-token log-likelihood shift for one recorded attempt (illustrative numbers, not from the paper). Card A raises the likelihood of the successful moves by 0.6 and lowers the likelihood of the failed moves by 0.2. The score is half the average d over successes minus half the average d over failures, so for card A it is 0.5 × 0.6 + 0.5 × 0.2 = +0.4, and it is injected. Card B lowers the successful moves by 0.3 and raises the failed moves by 0.1, giving 0.5 × (−0.3) + 0.5 × (−0.1) = −0.2. Card B is the best candidate on this problem and still negative, so the coach abstains. This follows the mixed-group case of the paper's Section 4.3; the numbers are made up to show the sign logic.

The exam-day row matters because the skills are training scaffolding, not a serving feature: at deployment the Generator and every skill are removed. On WebShop with Qwen3-8B the paper reports 45.1 for SkillScaffold with no skills, against 41.8 for the strongest skill-conditioned baseline run with its skills and 27.7 for that baseline once its skills are taken away. The ablations agree with the gate story: removing abstention drops ALFWorld from 94.8 to 85.8, and random selection drops it to 79.9. The scheme also works where it should. On ALFWorld, groups where the Scouts all failed gained 21.1 points of success after injection, while groups where they all succeeded changed by −2.3 and rarely received a skill. These are one paper's results on one 8B model family, so treat the gains as evidence for the mechanism, not as a general ranking. Related ideas: the four context fixes treat skills as something to load carefully, and SkillScaffold applies the same caution to training.

Goes deeper in: AI Agents → Tool Use & Function Calling → Skills as a Primitive

Related explainers

The Router Within: Gavel — the same question, which skill to use, answered at inference time instead of in training.
Matched replay gate for procedural memory — another case of testing a change against a fixed replay before accepting it.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based