Metis-Core Implements Asymmetric Multi-Head AlphaZero: Dual Drones Learn Pincer Attack Through Self-Play

The most fascinating moments in reinforcement learning often aren't when a model racks up high scores within human-defined rules, but when it spontaneously emerges with strategies the trainer never taught it. The release of Metis-Core v0.4.0-alpha delivers a rather convincing example: two "kamikaze" drones learned to execute a pincer attack on a target through self-play, with no scripted coordination whatsoever.

What Is Asymmetric Multi-Head AlphaZero

According to developer felixromo314's release post on Reddit, Metis-Core has added an implementation of the "Asymmetric Multi-Head AlphaZero" algorithm, accompanied by a C++ example built on Visual Studio 2022: KamikazeDroneSwarm.exe.

The "asymmetric" aspect refers to the fact that the two sides in a match have different action spaces, objectives, and even role definitions. Traditional AlphaZero-style methods are mostly used for symmetric games (such as Go or chess), where both sides follow identical rules and merely occupy opposing positions. In this scenario, however, one side is a beskar-armored Imperial shuttle equipped with a rotating laser turret, tasked with reaching the Death Star; the other side consists of two Rebel kamikaze drones whose mission is to intercept the shuttle before it arrives.

The "multi-head" aspect is reflected in the network simultaneously outputting multiple decision heads, allowing the framework to model different teams and different numbers of agents separately, which provides a more natural interface for multi-agent adversarial scenarios.

AlphaZero is a reinforcement learning algorithm proposed by DeepMind in 2017. It surpassed top human players in Go, chess, and shogi starting from zero knowledge, using only self-play. Its core technical combination includes: Monte Carlo Tree Search (MCTS) for prospectively evaluating moves during decision-making, and a deep neural network that simultaneously outputs a policy head (prior probabilities for each action) and a value head (expected win/loss for the current position), which together guide the search direction. In the training loop, the network continuously learns from MCTS search results to update its parameters, and the updated network in turn improves the quality of MCTS, forming a self-reinforcing flywheel. A key assumption of classic AlphaZero is symmetry: in board games, both sides follow identical rules, so a single network can play both roles simultaneously, merely with opposing perspectives. Metis-Core's "asymmetric" modification breaks this assumption, enabling the framework to handle scenarios where the two sides have entirely different action spaces and roles.

How the Pincer Attack Emerged

What's noteworthy is that the drones were not programmed with any coordination logic. The developer explicitly states that the pincer maneuver itself was not explicitly encoded into the reward or policy. The terminal reward remains only three tiers: +1 / -1 / 0, corresponding to successful interception, failure, or a draw.

Under such sparse rewards, the two drones learned to cooperate with each other, approaching the shuttle from different directions, forcing the turret to switch back and forth between the two sides and divide its attention — this is precisely the classic pincer tactic. A single drone charging head-on would almost certainly be shot down by the turret, but two drones applying simultaneous pressure from angled directions significantly increases the probability that at least one breaks through.

This coordination does not come from central scheduling; rather, each agent gradually discovers through self-play that "splitting up" yields higher expected returns. For multi-agent reinforcement learning, this is an intuitive and powerful demonstration.

Material Heuristic: A Pragmatic Trade-off for Accelerating Training

The developer admits that the drones are not entirely without prior guidance. He used a lightweight "material heuristic," enabled via setMaterialHeuristicWeight(0.3), with the weight set to 0.3.

This heuristic blends hand-designed positional evaluations — including proximity to the target, the spacing between the two drones, and the turret's exposure angle — with the network's value head, working together during the MCTS search process. In other words, it doesn't tell the drones to "pincer"; it merely provides rough signals about "proximity, dispersion, and exposure."

The developer emphasizes that this function is optional: you can set it to 0.2, 0.1, or simply not call it at all, in which case the heuristic is not activated and training relies entirely on the network itself. This design separates the engineering technique of accelerating convergence from the purity of the algorithm itself, allowing users to weigh the trade-off based on their compute budget and patience.

Monte Carlo Tree Search (MCTS) is the decision engine of AlphaZero-style algorithms. Its basic idea is: starting from the current position, repeatedly perform the four steps of "selection—expansion—simulation—backpropagation," gradually building a game tree. At each selection step, the algorithm balances "exploiting" known high-value branches against "exploring" branches that haven't been sufficiently visited, typically using the PUCT formula to combine the network's output prior probabilities with accumulated value. In standard AlphaZero, the value evaluation of leaf nodes comes entirely from the neural network's value head. Metis-Core's "material heuristic" extends this: when MCTS evaluates leaf nodes, it not only uses the network value head but also fuses a hand-designed positional evaluation function. This is similar to the approach in traditional game engines that combines "evaluation functions" with "search," except here the network value head serves as the learned evaluation function, while the hand-crafted heuristic provides additional domain signals. The weight of 0.3 determines the proportion of the hand-crafted signal in the total evaluation, and developers can adjust it or even zero it out based on experimental results, finding their own balance between "accelerating convergence" and "maintaining algorithmic purity."

Framework Capabilities and Open-Source Strategy

Metis-Core allows users to create Blue and Red teams and add any number of agents to each team as needed, making it not limited to pairwise adversarial scenarios but oriented toward more general multi-agent settings.

In terms of distribution, Metis-Core is provided in binary form (.dll / .lib / .h), but the complete source code for the examples is publicly available. The developer states that this allows users to clearly see how the framework is used and build their own environments on top of it. For developers who want to quickly validate ideas without being bogged down by low-level implementation details, this is a balanced approach.

It should be noted that the developer himself admits the current version "definitely has bugs and plenty of things to fix," and actively invites user feedback. This means v0.4.0-alpha is better suited for early adopters willing to tinker and with debugging capabilities, rather than those seeking an out-of-the-box production environment.

An Intriguing Adversarial Scenario

The example also supports human intervention: you can personally control the turret and fight against trained drones (human in the loop). This serves both as a demo and as a test — when you as a human find it difficult to simultaneously track two drones diving in from different directions, you can intuitively feel just how formidable the emergent coordination strategy is.

From a broader perspective, this project extends AlphaZero-style self-play from single-agent, symmetric games to asymmetric, multi-agent domains. The emergence of the pincer attack demonstrates that when the environment is rich enough and rewards are sparse enough, agents can potentially discover tactics that human military or game theory has long since summarized. For researching multi-agent cooperation and competition, as well as automated strategy generation, it provides a lightweight and accessible experimental platform.

Metis-Core is implemented in C++, making it friendly for performance-sensitive scenarios; binary distribution lowers the barrier to entry, while the openness of the example source code ensures learnability. If you're interested in multi-agent reinforcement learning or emergent strategies from self-play, this v0.4.0-alpha version is worth trying out.