FireFly: An Open-Source Imitation Learning Framework That Teaches Game AI to Play Like You

An Open-Source Project That Makes AI "Learn How You Play"

Developer Andrew Ahn has shared his new project FireFly with the community — an open-source framework designed for game agent experimentation. Its core idea isn't to have AI figure out game rules from scratch, but to directly imitate human player actions.

As the author describes it, FireFly's current training pipeline centers on Imitation Learning / Behavior Cloning: while you play a game, the framework simultaneously records screen observations and your actual inputs (actions), then trains a policy based on this demonstration data. In other words, the more you play, the better it can replicate your playstyle and decision-making patterns.

This "watch humans first, then learn" approach tends to converge to usable behavior much faster than pure Reinforcement Learning (RL), which starts from random exploration — making it especially well-suited for games with complex controls and sparse rewards.

Perception Layer: Multiple Visual and State Acquisition Methods

Before a game agent can "learn to play," the first step is understanding the screen. FireFly integrates multiple approaches at the perception layer, covering everything from traditional image processing to deep learning:

• OpenCV: Traditional computer vision processing, suitable for basic image preprocessing and feature extraction.
• YOLO object detection: Identifies key objects in the frame such as enemies, items, and UI elements.
• U-Net segmentation: Performs pixel-level semantic segmentation for scenarios requiring fine-grained spatial understanding.
• Template tracking: Tracks specific targets using template matching.
• Game memory / Lua state reading: Reads accurate state data directly from game memory or Lua scripts, bypassing the error-proneness of visual recognition.

This multi-method design means developers can freely combine "reading from the screen" and "reading from memory" based on the characteristics of each game, striking a balance between accuracy and generalizability.

Automated Data Annotation: Graph-Based Data Collection

Both imitation learning and perception models rely heavily on high-quality labeled data, and manual annotation is often the most time-consuming part of the entire pipeline. One of FireFly's highlights is its built-in graph-based data collection mechanism.

The author notes that this mechanism can automatically generate annotated datasets for training perception models as the player plays normally. In other words, gameplay itself becomes the data collection process, reducing the burden of building a separate annotation pipeline. For models like YOLO or U-Net that require large numbers of labeled samples, this "annotate while playing" approach can significantly lower the data barrier.

The "graph-based" collection mechanism refers to organizing game world states and events as a directed graph: nodes represent key game states or scene frames, while edges represent player actions or state transitions. This structure naturally captures temporal dependencies and makes it easy to query "what do humans typically do in a given type of scene?" Compared to simple time-series recording, the graph structure can more efficiently eliminate duplicate states, highlight rare but important edge cases, and provide stratified sampling by scene type for models like YOLO and U-Net — avoiding the common problem of over-sampling frequent scenes while under-representing rare ones.

The Challenge of Moving from Behavior Cloning to Reinforcement Learning

The author explicitly highlighted the feedback area he most wants to explore: how to extend a system like this from behavior cloning to reinforcement learning.

This touches on a classic limitation of imitation learning — behavior cloning can only learn the state distribution covered by demonstration data. Once the agent enters situations that human players rarely encounter, the policy tends to go off the rails, leading to compounding errors. Reinforcement learning allows the agent to improve itself through actual interaction with the environment and reward signals, breaking through the ceiling imposed by demonstration data.

Common evolution paths include: using the behavior cloning policy as a warm start for RL initialization, then fine-tuning through online interaction; introducing methods like DAgger to have an expert provide additional demonstrations at states where the agent makes mistakes; or going further with Inverse Reinforcement Learning (IRL) to infer a reward function from demonstrations. Whether FireFly will evolve along these directions is something the author clearly hopes to hear from the RL/IL community.

DAgger (Dataset Aggregation) is an interactive imitation learning algorithm proposed by Stéphane Ross et al. in 2011, specifically designed to mitigate compounding errors in behavior cloning. The core idea is: while the agent executes its policy, record the states it actually reaches in real time, then ask an expert (i.e., a human player) to re-annotate correct actions for those states, mixing the new data back into the training set for another iteration. This way, the state distribution of the training data gradually shifts toward the agent's own visitation distribution, rather than being forever confined to the expert's "ideal trajectories." Inverse Reinforcement Learning (IRL) takes a different route: rather than directly imitating actions, it infers the implicit reward function from expert demonstrations, then uses that reward function to drive standard reinforcement learning. Representative methods include Maximum Entropy IRL (MaxEnt IRL) and Generative Adversarial Imitation Learning (GAIL). For frameworks like FireFly, the appeal of IRL lies in the fact that once a reward function has been learned, the agent can optimize autonomously in scenarios the expert never encountered — with far greater generalization than pure behavior cloning.

Value for Developers

For anyone who wants to experiment with game AI hands-on, FireFly provides a relatively complete starting point: synchronized recording, diverse perception backends, automated data collection, and an open-source extensible architecture. It can serve as a practical platform for learning imitation learning workflows, as well as a toolkit for exploring perception model training.

The project is currently open-sourced on GitHub (AndrewAhn1000/FireFly), and the author has explicitly stated that feedback on architectural design and IL→RL extension ideas is welcome. For early-stage open-source projects like this, early community participation often determines how far they go — if you're working at this intersection, it's worth a look.

Note: This article is based on a single community post from the author. The actual effectiveness, performance, and maturity of the framework still await further independent validation.