MAPPO
An extension of the PPO algorithm to multi-agent settings, following the Centralized Training with Decentralized Execution (CTDE) paradigm. During training, agents share global information; during execution, each agent acts based on local observations. MAPPO has demonstrated high effectiveness in environments such as StarCraft and multi-robot collaboration.
Timeline (last 90 days)
Shared rewards combined with a centralized critic usually work well, but there is a free-rider risk; the author explicitly injects cooperation priors through reward terms such as focus fire and crossfire.
MAPPO adopts the CTDE paradigm: each agent has its own actor, but during training they share a centralized critic that can observe the global state to reduce gradient variance, while the actor at deployment time still relies only on local observations.
MAPPO achieved performance close to or even exceeding off-policy methods on benchmarks such as StarCraft micromanagement (SMAC) with a relatively simple implementation, and therefore has become a common baseline choice in multi-agent cooperative tasks.
On Reddit, a developer shared an experiment using MAPPO to train a 4-person squad to play survev (an open-source surviv.io clone), and released a 40-second gameplay clip.
In the experiment, the four squad members share the same actor network, with agent id used as input to distinguish identities.
Privileged state (global enemy positions, full health overview, etc.) is fed only to the centralized critic, to reduce variance during training.
MAPPO is a multi-agent extension of PPO that employs a shared policy network and a centralized value function.
MAPPO is a multi-agent extension of PPO that uses a shared policy network and a centralized value function; it performs well on benchmarks such as StarCraft and has good training stability.
All Facts (9)
Shared rewards combined with a centralized critic usually work well, but there is a free-rider risk; the author explicitly injects cooperation priors through reward terms such as focus fire and crossfire.
50%UnverifiedIn the experiment, the four squad members share the same actor network, with agent id used as input to distinguish identities.
50%UnverifiedPrivileged state (global enemy positions, full health overview, etc.) is fed only to the centralized critic, to reduce variance during training.
50%UnverifiedMAPPO adopts the CTDE paradigm: each agent has its own actor, but during training they share a centralized critic that can observe the global state to reduce gradient variance, while the actor at deployment time still relies only on local observations.
50%UnverifiedMAPPO achieved performance close to or even exceeding off-policy methods on benchmarks such as StarCraft micromanagement (SMAC) with a relatively simple implementation, and therefore has become a common baseline choice in multi-agent cooperative tasks.
50%UnverifiedMAPPO is a multi-agent extension of PPO that employs a shared policy network and a centralized value function.
50%UnverifiedMAPPO is a multi-agent extension of PPO that uses a shared policy network and a centralized value function; it performs well on benchmarks such as StarCraft and has good training stability.
50%Unverified开发者正在推进MAPPO（多智能体PPO）支持，目前基于共享策略的多智能体推理已可使用
50%UnverifiedOn Reddit, a developer shared an experiment using MAPPO to train a 4-person squad to play survev (an open-source surviv.io clone), and released a 40-second gameplay clip.
50%