Unity 6 Pure Neural Network Drone Swarm: Goodbye NavMesh

In game engines, implementing flight AI has long relied on two mainstream approaches: using flat NavMesh for path patching, or moving along pre-drawn spline tracks. Once real 6-DOF momentum, inertia, and vertical obstacles are introduced, traditional pathfinding algorithms almost immediately fail. A developer on Reddit shared the "Wolfpack Protocol" project, demonstrating a different path: training a four-drone autonomous combat swarm using continuous action space PPO reinforcement learning, with inference running entirely on Unity Sentis, completely abandoning NavMesh and pre-baked paths.

Core Architecture: Zero Python Runtime and Sentis Inference

The most noteworthy engineering decision in this project is the complete decoupling of training and inference. After PPO policy training is complete, it is directly deployed to Unity Sentis for forward inference, with no Python process dependency at runtime. The developer reports latency of sub-0.1ms—for real-time combat scenarios requiring high-frequency decisions, this magnitude means neural network evaluation hardly constitutes a performance bottleneck.

Traditional approaches often require building a communication bridge between the training environment and the game engine, with inference results transmitted via inter-process communication before feeding back into game logic, increasing both latency and deployment complexity. The Sentis approach embeds the entire policy network inside the engine, simplifying the deployment pipeline and enabling high-frequency calls for multiple agents.

PPO (Proximal Policy Optimization) is an on-policy reinforcement learning algorithm proposed by OpenAI in 2017, an improved version of policy gradient methods. It limits the deviation between new and old policies during each update by clipping the importance sampling ratio, achieving a good balance between training stability and sample efficiency. PPO has become one of the de facto standard algorithms for continuous action space control tasks (such as robot control and drone flight) due to its simple implementation, insensitivity to hyperparameters, and ease of parallelization. In game AI, PPO is often used to train agents requiring fine continuous control, such as the thrust and attitude adjustments of drones in this project.

Unity Sentis is Unity's official cross-platform neural network inference library (formerly Barracuda), allowing developers to import ONNX format models directly into Unity projects and execute forward inference on CPU or GPU at runtime. Sentis supports running on editor, desktop, mobile, and even WebGL without external services or Python processes. Its design goal is to enable trained models to seamlessly embed into game logic, particularly suitable for real-time AI scenarios requiring low latency and high-frequency calls. Sentis's inference backend automatically selects available computing devices and is optimized for Unity's job system.

38-Ray Spatial Whiskers: Emergent Obstacle Avoidance Without Retraining

The swarm's environmental perception doesn't rely on depth cameras or precomputed reachability graphs, but rather each drone projects 38 rays into 360-degree space, forming so-called "spatial whiskers." These rays simultaneously detect dynamic obstacles and teammate positions, with outputs fed directly as observation inputs to the policy network.

Interestingly, the developer emphasizes "emergent obstacle avoidance without retraining." This means the policy network learns not fixed paths for specific maps during training, but a general reactive capability to ray distance patterns. When environmental layouts change, the perception layer's data distribution still falls within the range the policy has seen, and obstacle avoidance behavior naturally generalizes. This is precisely the value of using perception rather than maps as input—maps changing requires retraining, ray readings changing do not.

Raycasting is a common technique in game engines for detecting ray intersections with scene geometry, returning hit distance, normals, and collider information. In AI perception, a set of rays distributed in a fan or spherical pattern is often called "whiskers," inspired by insect antennae. Compared to depth cameras or LiDAR, ray whiskers have extremely low input dimensionality (only one distance value per ray), low computational overhead, and are insensitive to lighting and texture changes, making them easier to train and generalize in small-scale games. In this project, 38 rays cover 360-degree space, detecting both static obstacles and dynamic teammates, forming a compact encoding of the surrounding environment.

Emergent Behavior refers to complex behaviors that agents spontaneously develop through interactions of simple rules during training, without explicit programming. In reinforcement learning, when policy networks are optimized with perception data as input and reward functions as guidance, they may learn obstacle avoidance or collaboration patterns unforeseen by designers. In this project, "emergent obstacle avoidance without retraining" means the policy has learned general reactions to ray distance patterns rather than memorizing paths for specific maps, so when environmental layouts change, as long as ray reading distributions remain within the training distribution, obstacle avoidance behavior can naturally transfer.

Tactical Swarm Command: Decoupled C# Coordination Layer

Individual drone flight decisions are handled by neural networks, but formation tactics—"who attacks, who covers"—are managed by an independent C# coordinator. This "Tactical Swarm Director" evaluates line-of-sight ray detection results and dot product calculations of player blind spots, dynamically rotating attackers into suicide dives while other units maintain high-altitude vigilance.

This layered design deserves special mention. End-to-end reinforcement learning could theoretically learn cooperative tactics, but training difficulty and interpretability are issues. Separating tactical decisions from neural networks and managing formation roles with deterministic rule code preserves the neural network's generalization capability for individual flight while making formation behavior debuggable and intervenable. In game AI engineering, this hybrid architecture of "neural for low-level, script for high-level" is typically more practical than pure end-to-end.

Hierarchical Architecture in game AI refers to dividing decision systems into different abstraction levels, typically with high-level responsible for tactical planning and low-level for specific execution. This design draws from behavior trees and hierarchical state machines, but in this project the high level uses deterministic rule code (C# coordinator) and the low level uses neural networks. The benefit of layering is that neural networks only need to handle continuous control problems they excel at, avoiding sparse rewards and training instability faced when learning cooperative tactics end-to-end; meanwhile, high-level logic is debuggable and intervenable, making it easier for designers to adjust formation behavior. This hybrid "neural for low-level, script for high-level" pattern is increasingly common in industrial game AI, especially suitable for multi-agent systems.

Why NavMesh Doesn't Work in 3D Aerial Combat

NavMesh is essentially a walkable area partition on a 2D manifold, assuming agents move along the ground with turning radii constrained to a plane. Aircraft have vertical degrees of freedom, momentum changes from diving and climbing, and mutual occlusion relationships in 3D space—none of which have representation in NavMesh.

Pre-drawn spline tracks are the other extreme: behavior is completely predictable, and once players learn the track patterns, combat becomes memorization. The value of this project lies in proving that real-time solutions based purely on physics and neural perception are already operational within engines, not just conceptual in papers. The developer positions it as "100% real-time physics and neural perception," with code open-sourced on GitHub's GeistForge-Engine repository.

NavMesh (Navigation Mesh) is a widely used pathfinding data structure in game AI that partitions walkable areas into convex polygon sets and builds navigation graphs for A* and other path searches. NavMesh's core assumption is that agents move on 2D planes with turning radii constrained to planar geometry. For aircraft, vertical degrees of freedom, momentum changes from diving and climbing, and mutual occlusion relationships in 3D space cannot be expressed in standard NavMesh. Although extended 3D navigation meshes or voxel navigation solutions exist, they typically require precomputation and struggle with high-speed dynamic obstacles, making them unsuitable for real-time aerial combat scenarios.

Spline Track is another common flight path control method, defining movement trajectories through pre-drawn Bezier curves or Catmull-Rom splines. Its advantages are completely predictable behavior and easy debugging, but the disadvantage is lack of flexibility—once players master the track patterns, combat becomes memorization. In swarm combat requiring autonomous decision-making and dynamic obstacle avoidance, spline tracks cannot provide sufficient reactive capability, so this project chooses a real-time solution based entirely on physics simulation and neural perception.

Implications for Game AI Development

From an engineering perspective, this project validates several reusable ideas. Sentis reduces inference latency to negligible levels, demonstrating that embedded neural network inference in engines can already meet real-time multi-agent requirements; low-dimensional perception inputs like ray whiskers are easier to train and generalize than image inputs, suitable for small to medium teams; hierarchical architecture lets neural networks handle only continuous control they excel at, with tactical logic delegated to traditional code, reducing overall system uncontrollability.

For indie developers, the open-source repository provides a starting point. The difficulty of deploying reinforcement learning in game AI has never been the algorithm itself, but the gap between training environments and runtime environments—this project uses Sentis to bridge most of that gap.