METIS-Core: A Pure C++ Reinforcement Learning Framework That Eliminates Python Overhead for Production

Developer Felix Romo shared an open-source project on Reddit called METIS-Core — a deep reinforcement learning (DRL) and multi-agent reinforcement learning (MARL) framework written entirely in C++. Its core value proposition is straightforward: remove Python completely from the training and inference execution loop, and call PyTorch's native C++ interface, LibTorch, directly.

For researchers and engineers who have long worked in Python-based RL ecosystems, this is a direction worth paying attention to. It's not trying to replace Ray or Stable-Baselines3 — instead, it targets a specific pain point: how costly is Python's runtime overhead when trained agents need to be deployed into production systems, high-stakes simulations, or edge devices?

Why Build a Native C++ RL Framework?

The vast majority of reinforcement learning research happens in the Python world — Ray, Stable-Baselines3, and PyTorch's Python frontend form the mainstream toolchain. Python's strengths lie in development efficiency and ecosystem richness, but the author points out that once autonomous agents are pushed into real production environments, Python's runtime overhead and the context-switching latency from inter-process communication (IPC) become serious bottlenecks.

This is especially true in real-time control scenarios. Applications like robot control, high-frequency trading, and edge inference often demand sub-millisecond response latency, and the overhead of the Python interpreter, GIL limitations, and the cost of cross-language data transfer can all cause a model that performs well in the lab to fall apart at deployment. METIS-Core's approach is to run the entire training and inference pipeline in C++, bypassing these structural issues.

Core Technical Features

METIS-Core's design revolves around several key principles.

Zero Python Overhead and Native LibTorch Integration

The framework is built directly on top of PyTorch's C++ API (LibTorch), fully leveraging its tensor operations and automatic differentiation engine to train neural networks. This means developers can still enjoy the mature capabilities of the PyTorch ecosystem, but without the Python interpreter involved at execution time. The author emphasizes that this design is specifically aimed at systems with hard requirements on real-time performance and low latency.

LibTorch is PyTorch's official C++ frontend library. It wraps PyTorch's tensor operations, automatic differentiation (autograd), neural network modules (nn::Module), and CUDA acceleration into a pure C++ API — no Python runtime required. Developers can use it to build, train, and run inference on neural networks, and exported model files (in TorchScript format) are fully compatible with the Python side, making interoperability between the two environments straightforward. LibTorch's primary use cases are exactly the environments where Python dependency is a concern: game engines, industrial controllers, and embedded systems. It's worth noting, however, that LibTorch's C++ API documentation and community activity are significantly less robust than the Python frontend, and the debugging experience is considerably more cumbersome — an engineering cost that should be evaluated upfront when choosing this technical path.

Tensor-Based Abstraction Layer

METIS-Core adopts a mathematically decoupled design: agents only handle pure tensor states, and their "intelligence" is entirely independent of the specific application domain. This application-agnostic abstraction allows the same core logic to adapt to vastly different domains — robotics, trading, logistics, defense — requiring only an outer adaptation of state and action tensor representations.

Shared-Trunk Multi-Head Architecture

For multi-agent cooperative scenarios, the framework includes built-in support for a Shared-Trunk Multi-Head architecture — multiple agents share a single neural network backbone to learn joint coordination strategies. This is a common parameter-sharing approach in MARL research that improves sample efficiency and policy consistency in cooperative tasks.

Multi-Threaded MCTS AlphaZero Engine

The framework also implements a natively running AlphaZero self-play loop, with Monte Carlo Tree Search (MCTS) executed in a multi-threaded manner in C++. Placing computationally intensive and highly parallelizable search processes like MCTS in C++ should theoretically yield substantial performance gains over Python implementations.

AlphaZero is a general-purpose board game algorithm proposed by DeepMind in 2017. Its core idea combines a deep neural network (which simultaneously outputs policy distributions and position values) with MCTS, iterating through pure self-play without any human prior knowledge to reach superhuman levels. MCTS constructs a simulated search tree at each decision step: starting from the current position, it repeatedly performs four phases — selection, expansion, simulation, and backpropagation — using the neural network's policy output to guide node selection and its value output to replace random rollouts for evaluating position quality, thereby concentrating the search on high-value branches within a limited time budget. This process is naturally suited for parallelization — multiple threads can simultaneously expand different simulation paths — but requires handling concurrent writes to tree nodes and synchronization issues like virtual loss, which is precisely where C++ implementations have a clear advantage over Python.

Current Progress and Milestones

Based on version history, the project is still in alpha, but there's a clear iteration path:

• v0.3.0 / v0.3.1-alpha (AlphaZero and Multi-threaded MCTS): Integrates a C++ version of the AlphaZero self-play loop, validated on the Los Alamos Chess (6×6 board) benchmark environment. This environment was chosen to balance tactical complexity with local hardware efficiency.
• v0.2.0-alpha (MARL and Shared Trunk): Implements the multi-head neural network architecture and provides a reference scenario called "SwarmDefenseTIE," where multiple agents learn cooperative defense transport tactics against attackers.
• v0.1.2-alpha (DQN and Quick Start): Includes minimal single-file environments like "Treasure Hunter" and "PursuitPolice" to help new users get up to speed quickly.

This progression from DQN to MARL to AlphaZero covers the major paradigms in reinforcement learning — from single-agent to multi-agent, and from model-free to search-based methods.

Roadmap: PPO and Asymmetric MARL

The author has outlined two clear next steps.

The first is AlphaZero Asymmetric MARL — merging multi-head networks with MCTS for asymmetric scenarios where agents have completely different action spaces and objectives, such as adversarial interactions between heterogeneous entities. These scenarios are quite common in real-world game theory and adversarial modeling, and represent one of the harder challenges in current MARL research.

The second is adding Proximal Policy Optimization (PPO) to support continuous action spaces. This is especially critical for robotics and physics simulation — DQN and AlphaZero with discrete actions cannot directly handle high-precision continuous control, and PPO is one of the mainstream algorithms for continuous control in the industry.

PPO was proposed by OpenAI in 2017 and is currently one of the most widely used algorithms for reinforcement learning tasks with continuous action spaces. It is a policy gradient method that introduces a "clipping" mechanism to limit the magnitude of each policy update, ensuring training stability while avoiding large policy jumps — the primary flaw of earlier policy gradient methods like REINFORCE. Unlike DQN, which can only output discrete actions, PPO can directly parameterize continuous distributions like Gaussian distributions, naturally supporting high-dimensional continuous control quantities like robot joint angles and thrust magnitudes. PPO has achieved strong baseline performance on landmark tasks such as OpenAI Five (Dota 2) and MuJoCo physics simulation, making it a de facto standard algorithm in robotics and physics simulation. Once METIS-Core completes PPO integration, its application scope will expand significantly from board games to continuous control domains.

Significance and Caveats

METIS-Core addresses a real engineering pain point: the limitations of Python-first RL ecosystems on the deployment side. For teams that need to bring reinforcement learning to edge devices or real-time systems, a native C++ framework can eliminate a great deal of tedious model export, serialization, and cross-language bridging work.

That said, a balanced perspective is warranted. The project is currently an alpha-stage solo effort, benchmark validation is limited to small-scale environments like 6×6 chess, and it's quite a distance from "production-grade." LibTorch's C++ development experience, documentation quality, and community support are all variables that need to be watched. The author himself actively solicited feedback in the post, indicating the framework is still being actively refined.

For developers focused on RL deployment engineering, this project at minimum offers a valuable reference implementation worth tracking on GitHub.