KnowGPO is a reinforcement-learning method for training deep search agents. It turns each question into a checklist of atomic knowledge conditions, groups rollouts by which conditions they have satisfied, and rewards each step for the new conditions it satisfies.

How is KnowGPO different from GRPO and GiGPO?

GRPO gives one outcome-based score to a whole run. GiGPO adds step-level credit but groups steps by matching observations, which leaves most steps alone. KnowGPO groups steps by the **set of facts found**, so nearly every step has comparison partners.

Why not just reward the agent for retrieving similar pages?

Page similarity can hide large differences. The paper shows two steps with 0.987 retrieval similarity where one had satisfied 0 of 12 knowledge conditions and the other 8 of 12. Checking facts against the answer's skeleton is grounded in the task, not in how the text looks.

Does the agent need the checklist when it runs?

The paper builds the conditions from each training question's reference answer, and the verifier uses them during training rollouts. The reported benefit is a better trained policy; the paper does not describe the checklist as a requirement at deployment.

Agent· 2026-10-19

KnowGPO — knowledge-grouped process rewards — What does it mean?

Jargon

Deep search agent: An LLM that searches, reads pages, rewrites its queries and decides when to answer over many turns, instead of the one retrieve-then-read pass of basic RAG.
Credit assignment: Deciding which actions in a long run deserve praise or blame. With one score at the end, every step in the run gets the same verdict.
GRPO: Group-relative policy optimization. It samples several runs for the same question and scores each run against the group average, so no separate value network is needed.
Advantage: How much better than expected an action or run turned out, usually the reward minus the group mean, divided by the group spread. A positive advantage makes the policy repeat the action.
Process reward: A reward given for intermediate steps instead of only the final answer. Its risk is that the signal can be noisy or can be gamed.
GiGPO and IGPO: The two step-level baselines the paper compares against. GiGPO groups steps whose environment states match. IGPO, as the paper describes it, relies on the model's own confidence signals.
Atomic knowledge condition: One single-hop fact the answer depends on, such as one link in a chain of reasoning. A question gets a small list of them, built from its reference answer.
Verifier: An LLM that reads each new observation and marks which knowledge conditions it now supports.

The news. On October 15, 2026, researchers from Zhejiang University and Tencent posted KnowGPO. It is a training method for search agents built on an 8B model. The paper reports gains of 3.8 to 4.2 points over the outcome-only GRPO baseline and the step-level GiGPO and IGPO baselines on three search benchmarks, and an 8B model that rivals most 32B open-source search agents. Read the paper →

Picture a team scavenger hunt. The organizer gives every team the same checklist of facts to find: the city, the year, the founder, the birthplace. At the end, the judge announces only whether the final answer was right. If the final answer is wrong, a hunter who found three of the four facts gets the same zero as one who wandered off. That is the credit-assignment problem of a deep search agent: one terminal score is spread across every step of a long run, and the learner cannot tell the useful searches from the redundant ones.

The usual fix is a step-level reward, but it needs a way to compare steps. GiGPO compares steps that reach the same state, and in a search task the state is a pile of retrieved pages. Two runs almost never retrieve the same pages, so the comparison group for most steps contains only that step. The paper measured this on 55,571 steps from 5,922 rollouts: with observation matching, 79.4% of the groups held a single step, and only 51.8% of steps had any partner at all. A lone step has nothing to be compared with.

KnowGPO changes what "same state" means, and the checklist is the whole idea. Before training, each question is turned into a list of atomic knowledge conditions: single facts the answer depends on. The paper builds them from the question, its reference answer and one successful reference run. During a rollout, an LLM verifier reads each new observation and ticks the conditions it now supports. A tick is never removed, so the knowledge state is simply the set of boxes ticked so far.

Now the grouping is easy. Every step that starts from the same set of ticked boxes goes into one knowledge group, even if the runs reached it through different queries and different pages. Inside a group, each step is rewarded by the incremental knowledge gain: how many new boxes it ticked. Its advantage is that gain compared with the group's mean and spread, which answers a fair question: from the same starting knowledge, did this step find more than its siblings?

The final advantage mixes the two signals: A = (1 − ω) · A_episode + ω · A_step. The paper uses ω = 0.3, so the final answer still dominates and the step reward refines it. At ω = 0.5 the model got worse on three of four splits, because too much local reward pulls the agent away from the real goal.

Grouping anchor	Two steps match when…	Steps in a group of 2 or more	Steps left alone
Observation (GiGPO-style) · paper, Table 5	retrieved text has cosine similarity ≥ 0.9	51.8%	79.4% of groups
Action · paper, Table 5	the generated queries are similar	79.4%	72.0% of groups
Knowledge state (KnowGPO) · paper, Table 5	the same set of conditions is ticked	97.8%	19.0% of groups

Read the table with one caveat. The second column counts steps in groups of two or more, and the last column is the paper's singleton ratio, which counts groups. Both come from the paper's own measurement on one set of 500 queries, so treat them as the paper's evidence, not a general law. The qualitative case in the appendix shows why the knowledge anchor matters: two steps with a retrieval similarity of 0.987 look identical to an observation matcher, but one had ticked 0 of 12 conditions and the other 8 of 12.

Here is the arithmetic on one group (illustrative numbers, not from the paper). A question has four conditions. Three of the eight rollouts reach the same state, with k1 and k2 ticked. Their next steps tick +2, +0 and +0 new boxes. The group mean is 0.67 and the standard deviation (population form) is 0.94, so the step advantages are +1.41 for the productive step and −0.71 for each redundant one.

Suppose all three rollouts failed the question, with an episode advantage of −0.5 each. GRPO alone gives all three steps −0.5. With ω = 0.3 the mix is 0.7 × (−0.5) + 0.3 × step advantage: the productive step ends at about +0.07 and each redundant step at about −0.56. The run still failed, yet the one search that found two facts is now reinforced instead of punished.

What did the extra signal buy? The ablation in the paper separates the two ideas. Adding incremental knowledge gain to plain GRPO gave 1.0 to 2.8 points. Adding knowledge-state grouping on top gave a further 1.0 to 4.2 points, the largest on BrowseComp-ZH. Swapping the knowledge-gain reward for the discounted future return that GiGPO uses cost 3.0, 6.0 and 3.4 points on Xbench-2505, BrowseComp-ZH and GAIA. The reward and the grouping each carry weight, and they work best together. Against GRPO, KnowGPO gained 5.9 points on BrowseComp-ZH and 5.2 on GAIA, and it was the only method that improved on every split.

Two cost and behavior checks matter for anyone who would use it. The verifier runs asynchronously, and the paper reports the bookkeeping itself at under 0.08% of the training budget. On queries only KnowGPO solved it used 9.83 search turns against 11.55 for GRPO, yet it used its longest searches, 10.85 turns, on queries both methods failed. So the agent learned to stop when evidence is enough, not to stop early everywhere.

Keep the limits in view. The results are one 8B model from one SFT start on three benchmarks, and an LLM judge scores the answers. The conditions need a reference answer and a successful reference run, so the method applies where those exist. A gap to proprietary systems remains: the paper reports 23.5 points to Kimi-K2.5 on BrowseComp-ZH and 28.1 to Claude-4-Sonnet on GAIA. Read the paper →

Goes deeper in: AI Agents → Evals & Diagnostics → Error Analysis First

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based