A New Framework for Self-Orchestration Learning in Decentralized Agent Swarms

The capability of a multi-agent system depends not only on its individual members, but also on how they coordinate. However, existing multi-agent systems often train agents under a fixed orchestration structure, separating individual capability learning from collective behavior organization. Recently, a framework called "Market-based Post-training" (MPT) was proposed to enable decentralized agent swarms to simultaneously learn action policies and self-orchestration.

The Core Problem: Separation of Orchestration and Learning

Traditional multi-agent training typically presupposes a fixed coordination structure—such as who does what and when—and then optimizes individual action policies on top of it. This separation leads to two problems: first, agents cannot dynamically adjust their participation patterns based on the task; second, collective behavior is difficult to evolve alongside individual capabilities.

The core idea of MPT is to let agents express their willingness to participate through "bidding." Bidding determines who takes action and how collective outcomes provide local training credit. In this way, action policies and bidding behavior can be jointly learned without a central coordinator.

"Orchestration" in multi-agent systems refers to how agents organize and collaborate: who executes actions when, how information flows among members, and how subtasks are allocated and integrated. Traditional methods typically treat the orchestration structure as a hyperparameter or a manually designed fixed pipeline, then use reinforcement learning or imitation learning to optimize each agent's policy. This "fixed structure, learned parameters" approach originates from the centralized training paradigm and performs well on static tasks, but when tasks require dynamically changing the number of participants, roles, or division of labor, fixed orchestration becomes a bottleneck. MPT emphasizes "self-orchestration" because it turns orchestration itself into a learnable variable, allowing the swarm to simultaneously evolve action and coordination structures during training.

How the Market Mechanism Works

In MPT, each agent issues a bid at every step, indicating the strength of its willingness to participate in the current task. The system allocates action rights based on bids and feeds the final outcome back to participating agents as a training signal.

This design allows individual capabilities and participation patterns to adapt together. Bidding is not only a selection mechanism but also becomes an effective signal for action training—ablation experiments confirm that learned bidding provides better training results than fixed allocation.

The research team evaluated MPT on three tasks: embodied fabrication, robot navigation, and deep research. Results show that it outperforms monolithic post-training, and agent participation patterns indeed evolve with learning.

Bidding here borrows from mechanism design ideas in economics: each agent gives a "bid" based on its own state and its estimate of the current subtask, and the highest bidder gains action rights. Unlike auctions, bidding here is not only used for allocation but also as a credit assignment signal—the quality of the final outcome is fed back to the agents that participated in bidding, so they learn when to actively compete and when to yield. The advantage of this mechanism is decentralization: there is no need for a central coordinator to collect global information and make allocation decisions; agents simply bid based on local observations. Ablation experiments show that learned bidding signals drive policy optimization more effectively than random or fixed allocation.

Experimental Results and Key Findings

When using the same post-trained action policies, learned self-orchestration outperforms other coordination strategies. This means orchestration itself can be optimized independently of action policies, and learned orchestration is more adaptive.

In addition, the trained swarm exhibits three additional advantages:

• Fast adaptation to related tasks: Beyond the original training objective, the swarm can transfer more quickly to related tasks.
• Scaled inference advantage: The more agents at inference time, the better the performance.
• Robustness to packet loss: Larger swarms suffer smaller performance losses when agents drop out.

These results indicate that learned self-orchestration supports reuse, scaling, and robustness of decentralized agent swarms.

"Monolithic post-training" refers to training all agents end-to-end as a single whole, rather than learning individual policies separately and then combining them. The advantage of monolithic post-training is global consistency, but the disadvantage is difficulty in scaling: once agents need to be added or replaced, the entire model must be retrained, and it lacks modeling of individual differences. MPT's comparative experiments show that under the same data and compute budget, letting agents learn policies separately and coordinate through bidding not only achieves better task performance but also remains stable when the number of agents changes. This validates the effectiveness of combining decentralized training with self-orchestration.

Implications for Multi-Agent System Design

The significance of MPT lies in breaking the inertia of "define structure first, then learn policies." By making orchestration itself a learning objective, the system can spontaneously form collaboration patterns adapted to tasks and environments.

For application scenarios requiring dynamic teaming, fault tolerance, and scaling—such as robot swarms and distributed research assistants—this method provides a trainable path. The code and trained models are open-sourced on GitHub.

However, evaluation is still limited to specific tasks, and the generalization ability of the bidding mechanism in more open environments, communication costs, and comparisons with other coordination methods remain to be further studied.

Robustness to packet loss and scaled inference advantage are important engineering metrics for decentralized multi-agent systems. Robustness to packet loss refers to whether remaining members can continue completing the task when some agents unexpectedly exit or communication is interrupted; scaled inference advantage refers to whether system performance improves rather than saturating or declining as the number of agents at inference time increases. In MPT, since each agent bids independently, the system naturally does not depend on the continued presence of any fixed role; when a member exits, other agents can take over its action rights through bidding. Moreover, more agents mean richer bidding candidates, potentially leading to better allocation solutions. These characteristics give MPT practical deployment potential in robot swarms and distributed scenarios.

One-line Summary

MPT framework enables decentralized agent swarms to simultaneously learn action policies and self-orchestration through a bidding mechanism, improving collaboration and robustness.

Paragraph Summary

The capability of a multi-agent system depends not only on individuals but also on how they coordinate. Traditional methods train agents under a fixed orchestration structure, separating individual capability learning from collective behavior organization. The recently proposed "Market-based Post-training" (MPT) framework lets each agent express its willingness to participate through bidding; the system allocates action rights based on bids and feeds outcomes back to participants as training signals. In this way, action policies and bidding behavior can be jointly learned without a central coordinator. On embodied fabrication, robot navigation, and deep research tasks, MPT outperforms monolithic post-training and fixed coordination strategies, and exhibits fast transfer, scaled inference advantages, and robustness to packet loss. Ablation experiments confirm that learned bidding provides more effective training signals. This work offers a trainable new path for the design of decentralized multi-agent systems.