17 Million Mouse Trajectories: Unpacking the Machine Learning Value of Gaming Behavioral Data

One Gamer's Accidental Data Treasure

Recently, a League of Legends player shared an intriguing side project on Reddit: he built a tool to record his in-game mouse trajectories, clicks, keyboard inputs, and other actions, all synchronized with video footage. Over roughly 350 matches and several months, he accumulated 17 million mouse cursor positions and 670,000 click records.

The poster openly admitted he's not a machine learning practitioner, but he astutely recognized that this dense, structured behavioral dataset might hold some kind of value — and asked: does data like this mean anything for ML or other applications?

That seemingly simple question touches on an increasingly important topic: the potential value of human behavioral telemetry data in AI training.

Behavioral telemetry refers to automated, continuous collection and recording of behavioral signals generated as users interact with a system. Unlike traditional logs, telemetry emphasizes high-frequency, high-precision capture of micro-level operational details — mouse acceleration, keystroke intervals, and so on — signals that often contain latent information about user intent, cognitive state, and even physiological characteristics. In industry, companies like Microsoft and Google have long used telemetry data for product optimization. In academia, behavioral telemetry is widely applied in human-computer interaction research, cognitive load assessment, and user modeling.

reddit source: 17 million mouse cursor positions and 670k clicks from a few months of my league of legends games. U

Data Structure: A High-Quality Behavioral Telemetry Sample

Judging from the data schema the poster made public, the dataset is organized quite rigorously and comprehensively. Each game is recorded as a structured object containing the following key fields:

Basic Metadata

gameMode: Game mode (e.g., ranked solo queue)
champion: Champion played
result: Win/loss outcome
kda: Kills/Deaths/Assists
durationMs: Game duration in milliseconds
resolution: Screen resolution (1920×1080)
sampleRateHz: Sampling rate of approximately 63Hz

Behavioral Data Streams

The real value lies in the following time-series data streams:

positions: Timestamped mouse coordinates {t, x, y}
clicks: Click events with left/right button distinction {t, x, y, b}
keypresses: Key events with duration {t, k, d}
gameEvents: In-game events (e.g., deaths) {type, t}
abilitySnapshots: Ability state snapshots {t, Q, W, E, R, h}

A sampling rate of approximately 63Hz means 63 coordinate points are recorded per second — a precision more than sufficient for capturing the micro-dynamics of human actions. From a signal processing perspective, the Nyquist sampling theorem tells us that 63Hz can faithfully reconstruct motion signals up to ~31Hz, covering the full effective frequency range of human hand movement (physiological hand tremor typically concentrates in the 8–12Hz range). This precision far exceeds the sub-10Hz rates used in typical web user behavior analytics, making fine-grained applications like fatigue detection and micro-gesture analysis feasible. More importantly, all data carries millisecond-precise timestamps synchronized with video replay — giving the dataset a natural advantage for temporal alignment and multimodal fusion.

Machine Learning Applications for Gaming Behavioral Data

From a machine learning standpoint, mouse trajectory and behavioral action data like this opens up a rich range of application directions.

Behavior Cloning and Imitation Learning

The most direct application is Imitation Learning. Imitation learning is an important branch of reinforcement learning whose core idea is to have AI learn a policy by observing expert demonstrations, without needing to manually design a reward function. The most fundamental approach — Behavior Cloning — treats expert state-action pairs directly as supervised learning data. This idea was already practiced in the ALVINN self-driving system back in 1989, and in recent years has seen major breakthroughs in game AI: both OpenAI Five and DeepMind's AlphaStar made extensive use of human gameplay recordings for pretraining, significantly cutting down learning time from scratch.

By treating "game state" (ability cooldowns, event snapshots) as input and "player actions" (mouse movement, clicks) as labels, one can train a model that mimics a specific player's operating style — a direct parallel to behavior cloning in autonomous driving.

Player Modeling and Skill Assessment

Features like mouse trajectory smoothness, click frequency (APM), and reaction latency are natural quantitative indicators of player skill level. Researchers can use these to build player profile models that predict rank, fatigue state, or focus level. Such models have real-world demand in esports training support and anti-cheat detection.

Anomaly Detection and Anti-Cheat

Human mouse movement follows Fitts' Law, exhibiting characteristic curved acceleration/deceleration patterns, micro-tremors, and path nonlinearity — biomechanical signatures that are individually unique, forming what's known as "Mouse Biometrics." Automated scripts tend to produce unnaturally smooth straight-line trajectories, fixed reaction delays, or click frequencies beyond human physiological limits — these anomalous patterns are precisely the key to distinguishing real human input from scripted bots. Major anti-cheat systems like VAC (Valve Anti-Cheat) and EasyAntiCheat have already incorporated behavioral feature analysis into their detection pipelines. This mouse trajectory dataset can serve as positive samples for training such detection models.

Action Intent Prediction

By combining game events (e.g., death moments) with action sequences, one can also study player decision patterns at critical moments and explore action intent prediction — forecasting a player's next move before they actually click.

Limitations and Practical Considerations

Objectively speaking, this dataset also has several clear shortcomings worth carefully weighing.

Bias from a Single Player Sample

The biggest issue is that all data comes from a single player. Overfitting is a classic problem in machine learning — where a model excessively fits specific patterns in training data and loses generalization ability on new data. Single-player datasets are especially prone to this: the model may learn the player's personal quirks (such as habitually resting the cursor in a specific screen region) rather than universal gameplay strategy. Mitigation strategies include data augmentation (applying random affine transforms to coordinates), Dropout regularization, or Domain Adaptation techniques. The more fundamental solution is collaborative data collection — aggregating data from multiple players through an open client tool, which is also how companies like Riot Games build their internal datasets.

Missing Complete Game State

The data records personal actions and some events, but lacks complete situational information — enemy positions, minimap awareness, gold differentials, and other critical context are all absent. For a highly macro-dependent game like a MOBA, it's difficult to reconstruct the full decision-making logic from personal action data alone, which limits its value to some degree for training "strategic decision-making" AI.

Scale That Sounds Large but Is Actually Limited

17 million coordinate points sounds impressive, but it translates to only 350 games. For deep learning tasks that require massive, diverse samples, this volume qualifies only as a proof-of-concept dataset — still quite far from training a robust, production-grade model.

The Value of Behavioral Data Collection Through a Personal Project Lens

Setting aside the technical details, this case itself offers an interesting perspective: in the AI era, the barrier to collecting behavioral data is dropping rapidly, while the potential value of that data continues to rise.

This trend aligns closely with the core needs of the Embodied AI field. Embodied AI systems need to learn by interacting with physical or simulated environments, and their training is heavily dependent on high-quality real human behavioral data. In the current large model era, text data is relatively abundant, but fine-grained human action data — especially time-series data with complete context — remains extremely scarce and expensive. Projects like Google DeepMind's RT-2 and Figure AI have invested enormous sums to collect this kind of data. As a naturally digitized environment for recording human behavior, game telemetry data is seen as an important source for filling this gap — academics refer to such data as a "Digital Behavioral Substrate."

An ordinary player building a telemetry system in their spare time can produce a well-structured, multimodally aligned, high-quality dataset. This "self-collected data" model is one of the key approaches for obtaining real human behavioral data in cutting-edge fields like embodied intelligence and imitation learning.

For this Reddit poster, even if the dataset isn't immediately commercially viable, it's at minimum an excellent personal ML starting point: he could try training a simple agent that mimics his own playing style, or build visualization tools to analyze his personal gaming habits. More importantly, he now has a continuously extensible data collection pipeline — if more players can be recruited to contribute data, the problems of scale and diversity will resolve themselves.

Conclusion

This Reddit question, which looked like nothing more than an offhand share from a gaming enthusiast, actually reflects a corner of the AI data ecosystem that is increasingly receiving attention: fine-grained telemetry data of human operational behavior. It reminds us that valuable machine learning training data doesn't have to come from expensive professional annotation — it can also be born from every ordinary mouse click in daily life. For individual developers, the ability to recognize and structure these "digital footprints" may well be one of the doors that opens onto AI practice.