**WeAct** (Weight-level Internal Actions) is a method that trains separate LoRA adapters for different agent capabilities over a frozen backbone and lets the agent select them during a run through tool calls named `load_lora` and `offload_lora`.

Why not just merge the adapters into one?

In the paper, every tested static merge scored worse than WeAct on all eight benchmarks, and the best merge averaged 35.66% against 45.47% for WeAct-hard. The authors also found that merging changes each specialist's local effect, even when no update is discarded.

Does switching adapters make the agent slower?

Switching has a cost: adapter control and cache handling together took 2.8% of run time for WeAct-hard and 8.2% for WeAct-soft, and the paper's accounting includes a cache flush and a post-switch prefill. The authors still measured lower mean latency than full fine-tuning, 20.3% for hard and 33.4% for soft.

How does WeAct relate to serving many LoRA adapters?

Both rely on one frozen backbone with small swappable adapters. Multi-LoRA serving chooses an adapter per request; WeAct chooses one per stage inside a single request, which is why cache handling after a switch matters.

Agent· 2026-10-21

WeAct paper — Tool-called LoRA adapter switching — What does it mean?

Jargon

LoRA adapter: A pair of small low-rank matrices added to a frozen model's weight matrices; training only these changes the model's behaviour cheaply. See LoRA math at serving time.
Frozen backbone: The base model whose weights are never updated. Here every adapter is trained on top of the same frozen Qwen3-32B.
Catastrophic forgetting: When training a model on a new skill overwrites what it learned for an earlier skill, because both share the same parameters.
Static merging: Combining several trained adapters into one fixed update before deployment (methods named in the paper include TIES, DARE-TIES and TSPA). The merged weights are the same for every stage of a run.
Full SFT: Supervised fine-tuning of all the model's weights on demonstration data. It is the strongest single-weight-set baseline in the paper.
Weight-level internal action: The paper's name for an agent action that changes which weights generate the next tokens, as opposed to an external action such as a web search.
Tool calling: The model's native way of emitting a structured request that a harness executes and answers. See the tool schema step.
ReAct: The agent pattern that alternates reasoning with external actions (tool calls) and observations. WeAct adds a second kind of action to it.

The news. On October 19, 2026, researchers from Carnegie Mellon University and Duke University posted the paper WeAct: Dynamic Weight Switching as Agent Actions to arXiv. It trains separate LoRA adapters over a frozen Qwen3-32B and lets the agent select them mid-run through tool calls. The authors report a 6.2% relative gain over full fine-tuning and 27.5% over the strongest static LoRA merge on eight deep-research benchmarks. Read the paper →

Picture the drill. In the drill picture, one permanently fused bit is a compromise for every material. The paper's counterpart is an agent whose skills are all trained into one set of weights, or averaged into one merged adapter, so the same weights must plan, search and write (an illustration, not a result the paper states). WeAct keeps the drill body (the frozen backbone) and gives each stage its own bit, a capability-specific LoRA adapter trained only on that stage: LoRA-plan sets goals, LoRA-search gathers and checks evidence, LoRA-write turns evidence into an answer.

The new part is how the bit is chosen. The harness exposes two ordinary tools, load_lora and offload_lora, each taking an adapter name. Because the choice is a tool call, the model can reuse the tool-calling ability it already has, and the paper trains no separate router. The call and its result are written into the same conversation history, so the next tokens come from the newly loaded weights but still see everything that happened before. The paper tests two policies: WeAct-hard prescribes loading every adapter in the workflow, and WeAct-soft lets the model choose from the history. If you know how a tool schema is defined, this is the same mechanism pointed at the model's own weights instead of the outside world.

Method (Qwen3-32B)	Weights used during a run	Avg. accuracy, 8 benchmarks	Source
Best static merge	one fixed merged update	35.66%	paper, Table 1
Single LoRA	one adapter trained on everything	38.73%	paper, Table 1
Full SFT	all weights, one set	42.80% (derived: 45.47 − 2.67)	paper, Sec. 4.2
WeAct-soft	adapter chosen by the model per stage	43.21%	paper, Table 1
WeAct-hard	adapter prescribed per stage	45.47%	paper, Table 1

Why does separation help? The paper gives two reasons. First, training interference: when two skills share parameters, a gradient step for skill k changes skill j's loss by roughly minus the learning rate times the dot product of the two gradients, so a step that helps one skill can hurt another. With separate adapters, skill j's adapter receives zero gradient from skill k's loss, so training one skill leaves the others' parameters unchanged; the authors refresh a capability this way without data replay in their writing test. Second, fusion mismatch: even adding all the updates together without compressing them changed a specialist's local effect, because the other adapters' contributions leak into it. In the authors' tests no merge removed the loss degradation on any of the three capabilities, so they keep the specialists intact and select one at run time.

Separation also makes updates local. The authors added a new code adapter without retraining the others, and refreshed only the writing adapter to improve report quality. Adding or updating a capability touches one adapter and leaves the backbone and every other adapter unchanged. For an operator, that is closer to shipping one package than retraining one monolith, and it has the same shape as serving many adapters on one backbone.

Now the worked example, using the paper's reported numbers (Qwen3-32B, eight deep-research benchmarks, the same three-adapter library). The best static merge averaged 35.66%. WeAct-hard, using the same adapters but selecting one per stage, averaged 45.47%: +9.81 points, a 27.5% relative gain. Against full fine-tuning at 42.80%, the gain is +2.67 points, or 6.2% relative. Notably, the merge also scored below a single LoRA trained directly on everything (38.73%), so merging lost accuracy even before any comparison with WeAct. Switching is not free: the authors measured that adapter control plus cache handling together took 2.8% of run time for hard and 8.2% for soft, and the cache part is real because the paper's accounting includes a cache flush and a post-switch prefill. (illustrative: on a run lasting 200 seconds, that combined 8.2% is about 16 seconds.) Even so, mean answer-ready latency was 20.3% lower (hard) and 33.4% lower (soft) than full SFT.

Read the result with its limits in mind. Everything above comes from one backbone (Qwen3-32B), deep-research tasks, and a library of three capabilities defined by the authors' own stage segmentation, with results reported by the authors themselves. The paper does not show that the same gains hold for other model sizes or for agents whose stages are harder to separate cleanly. The code-adapter test also showed only small changes elsewhere (for example +1.0 on GAIA), so it supports that adding a capability does no harm more strongly than that it helps broadly.

Goes deeper in: AI Agents → Tool Use & Function Calling → The Tool Schema

Related explainers

CacheReforge — repairing stale KV caches after LoRA updates — what happens to cached K/V when the adapter changes under it
Uno's LoRA diffusion adapter for block-parallel decoding — another case of a small adapter changing how a frozen model behaves

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based