WeAct: Dynamic Weight Switching Lets AI Agents Invoke Capabilities on Demand

Long-horizon agents need to learn a variety of heterogeneous capabilities from interleaved trajectories, but shared parameter updates easily lead to catastrophic forgetting. WeAct proposes adding capability-specific LoRA adapter selection to the agent's action space, enabling on-demand capability composition through the native tool-calling interface. On eight deep research benchmarks, Qwen3-32B with WeAct achieves an average accuracy 6.2% higher than full fine-tuning and 27.5% higher than the strongest static LoRA merging method.

Problem: Fixed Parameter Configurations Struggle with Dynamic Tasks

Long-horizon agent trajectories often span multiple stages, and different stages require different capabilities—early stages may emphasize information retrieval, middle stages require code-based computation, and later stages rely on writing and synthesis. Traditional approaches either jointly train all capabilities or statically merge multiple LoRA adapters, but both maintain a fixed parameter configuration throughout execution.

The cost of this fixed configuration is obvious. When the model updates shared parameters to learn new capabilities, previously acquired capabilities may be overwritten, causing catastrophic forgetting. Static merging avoids training conflicts but cannot dynamically adjust capability weights according to the trajectory stage—it remains essentially a one-size-fits-all solution.

WeAct's core observation is simple: if an agent can already dynamically select external tools through the tool-calling interface, why not make "switching internal weights" an action as well?

Catastrophic Forgetting is a classic problem in continual learning for neural networks: when a model is trained sequentially on a series of tasks, updating shared parameters for a new task overwrites the weights that old tasks depend on, causing sharp performance degradation on old tasks. This phenomenon is especially pronounced in long-horizon agent scenarios, because the capabilities required at different stages (retrieval, computation, writing) differ greatly, gradient directions conflict during joint training, and shared parameters struggle to satisfy all capabilities simultaneously.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that freezes the backbone weights of a pretrained model and injects low-rank decomposition matrix pairs (A and B) as bypasses in specific layers, training only these two small matrices to adapt to downstream tasks. Since the number of trainable parameters is typically less than 1% of the original model, LoRA greatly reduces training costs, and different LoRA modules are naturally isolated from one another—this provides the foundation for WeAct's modular design.

Method: Incorporating Adapter Selection into the Action Space

WeAct (Weight-level Internal Actions) trains multiple capability-specific LoRA adapters on a frozen backbone model, with each adapter corresponding to one capability, such as retrieval, code-based computation, or writing. The key is that adapter selection is modeled as an internal action of the agent, on par with calling external tools.

These internal actions reuse the model's native tool-calling interface, so no additional routing module or controller is needed. In a shared context, the agent autonomously decides which adapter to activate based on the current task requirements, enabling on-demand capability composition. Throughout the process, the backbone parameters remain unchanged, and the adapters do not interfere with one another.

This design brings two direct benefits. First, capability learning and capability invocation are decoupled: adding a new capability only requires training a new adapter and does not affect existing capabilities. Second, adapter-selection actions and external tool calls share the same interface, reducing the complexity of the agent's decision logic.

Tool Calling / Function Calling is the standard mechanism by which large language models interact with external systems: during generation, the model outputs structured function names and parameters, which the runtime environment executes and returns results for. Mainstream models (such as the Qwen series) have internalized this protocol during training and can autonomously decide when to call which tool based on context. WeAct's key insight is that this interface is essentially an "action selector" that does not care whether what is selected is an external API or an internal weight module. Therefore, switching LoRA adapters can be encoded as a tool call, reusing the model's existing decision-making ability at zero additional overhead.

Experiments: Less Compute, Higher Accuracy

The research team evaluated WeAct on eight deep research benchmarks with Qwen3-32B as the base model. Results show that WeAct's average accuracy improves by 6.2% over the corresponding full fine-tuning approach while using less compute. Compared with the strongest static LoRA merging method, the improvement reaches 27.5%.

This comparison is noteworthy: although static merging methods also use multiple adapters, the merged weights are fixed and cannot change with the trajectory stage. WeAct's dynamic switching allows the model to invoke different capabilities at different stages, avoiding mutual dilution among capabilities.

From a compute perspective, freezing the backbone means most parameters do not require gradient updates, and training cost is concentrated on a small number of adapters. For agent systems that need to continuously expand capabilities, this modular approach is clearly more sustainable.

Static LoRA Merging refers to merging multiple separately trained LoRA adapters into a single weight set through methods such as weighted summation, Task Arithmetic, or TIES, and loading only the merged result during inference. Such methods avoid task conflicts during training, but the merged weights remain fixed throughout inference and cannot be dynamically adjusted according to the input or trajectory stage. When adapters corresponding to different capabilities differ greatly in direction in parameter space, simple merging causes mutual interference and capability dilution—this is exactly the root cause of WeAct's dynamic switching approach leading by a large margin in accuracy.

Modular Continual Adaptation: New Capabilities Do Not Harm Old Ones

WeAct also demonstrates modular continual adaptation. In the experiments, the team added a code-computation adapter to expand the agent's capabilities while also updating the existing writing adapter to improve report quality. Neither update modified the backbone model or other adapters.

This solves a practical challenge in continual learning: agents always need new capabilities after deployment, but each update should not degrade old capabilities. WeAct achieves this through adapter isolation—new capabilities are added incrementally, the adapters for old capabilities remain unchanged, and the backbone as a shared foundation is also unaffected.

For agent products that need long-term evolution, this property means capability modules can be iterated independently without retraining the entire model each time.

Limitations and Implications

WeAct's current validation focuses on deep research tasks and the Qwen3-32B base model. How the number of adapters and the granularity of capabilities should scale with task complexity is not sufficiently discussed in the original work. The adapter-selection action itself also requires the agent to learn to make correct decisions, which places demands on the quality and coverage of training data.

However, its core idea is general: change parameter configuration from a static preset into a dynamic action, allowing the agent to autonomously manage its own capabilities during execution. This is consistent with the development path of tool calling—moving from fixed workflows to on-demand composition. As model capabilities increasingly exist in modular form, how to schedule these modules is becoming a key issue in agent design.

One-line Summary

WeAct incorporates LoRA adapter selection into the agent's action space to enable dynamic capability composition, significantly surpassing full fine-tuning in accuracy.

Paragraph Summary

To address catastrophic forgetting when long-horizon agents learn multiple heterogeneous capabilities from interleaved trajectories, WeAct models capability-specific LoRA adapter selection as internal actions of the agent. These actions reuse the model's native tool-calling interface without requiring an additional routing module, allowing the agent to autonomously activate adapters for retrieval, code computation, or writing according to the task stage while backbone parameters remain frozen. On eight deep research benchmarks, Qwen3-32B with WeAct achieves an average accuracy 6.2% higher than full fine-tuning and 27.5% higher than the strongest static LoRA merging method, with less compute. The design also supports modular continual adaptation: adding a new capability only requires training a new adapter and does not affect existing capabilities or the backbone model.