Training a Neural Network to Play Tic-Tac-Toe with Minimax Data: A Data Quality Experiment

A Seemingly Simple Yet Insightful Experiment

Tic-Tac-Toe, a game with only 9 squares, has long been a classic introductory case for algorithm education. Its state space is small enough to be completely solved, making it naturally suited for validating various AI techniques. Recently, a developer shared their experimental project on Reddit: instead of using traditional search algorithms for direct gameplay, they trained a simple neural network to learn optimal moves from scratch.

The core idea behind this project isn't complicated, but it touches on a fundamental question in machine learning — how much does training data quality actually matter for small models? The author used a Minimax solver to generate "ground truth answers," then trained a neural network via supervised learning to imitate these optimal decisions. The entire program runs directly in the terminal, allowing users to play against the trained network.

Generating High-Quality Training Data with the Minimax Algorithm

The first step of the project is constructing high-quality training data. The author employed the classic Minimax algorithm as the "teacher."

Why Choose Minimax as the Data Generator

Minimax is a search algorithm for finding optimal strategies in zero-sum games. It recursively assumes the opponent always makes the move most favorable to themselves (and least favorable to us), thereby computing the best move for the current board position. The algorithm can be traced back to John von Neumann's foundational game theory paper published in 1928 and represents a cornerstone achievement at the intersection of game theory and artificial intelligence. The algorithm constructs a complete game tree where MAX layers represent our side trying to maximize gains, MIN layers represent the opponent trying to minimize our gains, and values are computed by backtracking from leaf nodes (terminal states) upward layer by layer.

For a game with an extremely small state space like Tic-Tac-Toe, Minimax can exhaustively enumerate all possibilities within milliseconds, producing theoretically flawless optimal solutions. The complete game tree for Tic-Tac-Toe contains approximately 255,168 terminal paths, but after removing symmetries and unreachable states, the actual number of independent positions to evaluate is around 5,478, making complete solving virtually instantaneous on modern hardware.

The author calls the Minimax solver for every reachable board state, computing the optimal move for that state. This forms a set of board state → optimal move mapping pairs that serve as supervised learning labels for the neural network. This approach of "using a known-correct algorithm to generate labeled data, then having a model learn from it" is essentially a form of knowledge distillation — transferring symbolic search wisdom into a continuous neural network.

Knowledge Distillation was formally proposed by Geoffrey Hinton et al. in 2015, with the core idea of compressing knowledge from a large, complex "teacher model" into a small, efficient "student model." Traditional knowledge distillation typically uses soft labels from the teacher model's output to transfer "dark knowledge" such as inter-class similarities. The approach in this project is a generalized form of knowledge distillation — the teacher isn't a neural network but a symbolic search algorithm, yet the underlying logic is identical: converting computationally intensive reasoning capabilities into a lightweight model's single forward pass, achieving orders-of-magnitude speedup at inference time.

Determinism and Completeness of Training Data

Interestingly, data generated this way possesses determinism and completeness: since Tic-Tac-Toe's state space is finite, it's theoretically possible to cover all legal positions. This means the model faces a nearly "complete" training set, with virtually no out-of-distribution (OOD) concerns.

The out-of-distribution (OOD) problem is one of the core challenges in machine learning. In most real-world applications (such as autonomous driving and medical diagnosis), when models encounter inputs significantly different from the training data distribution, they often produce unreliable predictions. Completely covering all possible inputs is nearly impossible in practical scenarios, but Tic-Tac-Toe's finite state space makes "zero OOD" achievable — something extremely rare in real machine learning problems. Precisely because of this, we can study the pure effects of model capacity and data quality on learning outcomes without the interference of generalization error, which is one reason this teaching case is particularly valuable.

Transforming a Game Problem into a Neural Network Classification Task

With data in hand, the author treats it as a standard supervised classification problem.

The neural network's input is the encoded board state (typically a 9-dimensional vector representing the occupancy of each square), and the output is a probability distribution over the 9 squares indicating the best move position. The model learns to imitate Minimax's optimal decisions through classification loss (such as cross-entropy).

Regarding board state encoding, there are multiple options available. The simplest uses a 9-dimensional vector where each position is represented by -1 (opponent's piece), 0 (empty), or 1 (own piece). A more refined approach uses one-hot encoding, representing each square with a 3-dimensional vector for three states, forming a 27-dimensional input vector. Some researchers use dual-channel binary matrices, which better align with convolutional neural network input formats. The choice of encoding directly affects the difficulty of network learning, as it determines the geometric structure of the input space and the complexity of decision boundaries.

The elegance of this modeling approach lies in how it bypasses the complex reward design and exploration problems in reinforcement learning. Since we already have optimal action labels for every state, there's no need for the model to figure things out through trial and error — we can simply train it as a "look at the board, pick a square" classifier. For problems with small state spaces where optimal solutions can be completely solved, this is often the most direct and efficient path.

In the game AI domain, supervised learning (imitation learning) and reinforcement learning represent two fundamentally different training paradigms. Supervised learning requires pre-existing expert data, with stable training and fast convergence, but model capability is capped by data quality. Reinforcement learning autonomously discovers strategies through trial-and-error and reward signals, theoretically able to surpass any known expert level, but faces the exploration-exploitation dilemma, reward sparsity, and training instability. For problems like Tic-Tac-Toe where optimal solutions are known and exhaustively computable, supervised learning is undoubtedly the more economical choice.

Of course, this also means the model's capability ceiling is firmly locked by the Minimax data: it can at best approach the teacher algorithm's level, never surpass it. This forms an interesting contrast with AlphaGo's early path of using supervised learning to imitate human game records, then breaking through human-level performance via self-play. AlphaGo's development history perfectly illustrates this trade-off: AlphaGo Lee first used supervised pre-training on human game records for good initialization, then improved through reinforcement learning self-play; the subsequent AlphaGo Zero completely abandoned human data, learning purely from self-play, ultimately achieving a higher level — proving that when problems are sufficiently complex, human data can actually become a capability bottleneck. However, for problems like Tic-Tac-Toe where the optimal solution is the theoretical ceiling, surpassing the teacher is meaningless by definition.

Data Quality Control Experiment: Random Data vs. Optimal Data

The most valuable part of the project is actually the author's planned follow-up.

They plan to train a second version of the model, this time using data generated from random self-play rather than Minimax optimal data, then compare the performance differences. This experimental design directly addresses a core proposition:

For a small model like this, how much does training data quality actually matter?

Why This Control Experiment Matters

Data generated from random self-play will contain a large number of suboptimal or even incorrect moves. If trained on this "low-quality" data, what will the model learn? Will it be led astray by noise, or can it extract some statistical patterns from the large volume of data?

This question holds practical relevance in today's era of large models. The industry widely emphasizes that "data quality trumps data quantity," and this Tic-Tac-Toe experiment provides a controllable, interpretable, and fully verifiable miniature sandbox.

The data quality question has been elevated to unprecedented importance in the large language model era. Technical reports from Meta's Llama series, Google's Gemma, and other mainstream models consistently emphasize that carefully curated high-quality datasets often improve model performance more than simply increasing data volume or model parameters. Notable examples include: Microsoft's Phi series models, which used "textbook-quality" synthetic data to train small models that performed far beyond expectations for their parameter count; and the substantial resources Anthropic and OpenAI invest in quality control of human feedback data during the RLHF (Reinforcement Learning from Human Feedback) phase. These phenomena all point to the same conclusion: in an era where model architectures are converging, data quality and data engineering are becoming the key battleground for differentiation.

Because Tic-Tac-Toe's optimal solution is known, we can precisely measure how far each model deviates from the optimal strategy, thereby quantifying the impact of data quality. This is far clearer than running similar ablation experiments on models with tens of billions of parameters — in large model experiments, data quality effects are often entangled with model scale, training duration, learning rate schedules, and other factors, making isolation difficult. The minimalism of the Tic-Tac-Toe experiment precisely eliminates these confounding variables.

Machine Learning Insights Behind a Small Project

The project itself isn't technically sophisticated — it doesn't use complex network architectures or cutting-edge algorithms — but its value lies precisely in "seeing the big picture through the small."

First, it clearly demonstrates how to transform a game problem into a trainable supervised learning task, making it an excellent teaching case for understanding knowledge distillation and imitation learning. Second, the author's proposed data quality control experiment grounds an abstract ML principle in a concrete, empirically verifiable scenario.

For beginners, complete, concise, and runnable open-source projects like this one (code published on GitHub) build intuition more easily than reading lengthy papers. The author's open attitude in actively soliciting feedback on "data generation structure" improvements also reflects good engineering practice — thinking clearly about where data comes from and how to ensure its quality before writing code is often more critical than tuning the model itself.

From a broader perspective, this project also reflects an increasingly valued concept in AI engineering: Data-Centric AI. Strongly advocated by Andrew Ng since 2021, this paradigm shift argues that in an era where model architectures and training techniques are relatively mature, systematically improving data quality is the most effective lever for enhancing AI system performance. Though the Tic-Tac-Toe project is tiny, its embodied philosophy of "get the data right first, then train the model" is perfectly aligned with this industry-level methodology.

If you're interested in the underlying mechanisms of machine learning, consider starting with a small project like this where you can play against it yourself and understand every line of code. Sometimes the simplest experiments reveal the most profound principles.