AI Agent Runs a Fully Automated Chess Channel: A Real-World Practice of Combining LLMs with Engines

The "Holy Grail" of Chess Programming: Making AI Explain Moves Like a Coach

Chess engines have dominated the game for decades—they can compute the best move, yet they've never been able to "explain" why. Modern top-tier engines like Stockfish and Leela Chess Zero represent the two mainstream technical paradigms in today's computer chess: Stockfish uses Alpha-Beta pruning search combined with NNUE neural network evaluation, while Leela relies entirely on Monte Carlo Tree Search (MCTS) plus deep neural networks, trained through self-play reinforcement learning.

Alpha-Beta pruning is an optimized version of the minimax algorithm. By terminating early any search branch that cannot possibly yield a better result, it compresses the effective search depth from exponential to approximately square-root order. On top of this, modern Stockfish stacks dozens of optimization techniques such as null-move pruning, search extensions, and transposition table caching, reaching search depths of 20 plies or more. The evaluation function supporting this search capability is precisely the NNUE (Efficiently Updatable Neural Network) architecture—an efficient neural network first pioneered by Japanese shogi programmer Nodchip in 2018 and later introduced into chess by the Stockfish team. Its core innovation lies in the "incremental update" mechanism: each time a move is made, the board change typically involves only a few squares, and NNUE exploits this sparsity by updating only the affected network activations rather than recomputing the entire network. Its inference speed is two orders of magnitude faster than traditional deep neural networks and can run in real time entirely on a CPU, making Stockfish a representative of the "hybrid paradigm" that combines traditional search depth with neural network evaluation precision.

Both types of engines have ELO ratings far exceeding 3500, well beyond any human player—and the ELO rating system was designed by Hungarian-American physicist Arpad Elo. Its mathematical essence is a probability prediction model based on the logistic function: before a game, the win probability is predicted from the rating difference, and after the game, the deviation between the actual result and the expected result multiplied by the K-factor gives the rating change. For every ~200 point increase in rating difference, the higher-rated player's win probability increases by about 75%, but this relationship follows a logistic curve rather than a linear one. This means that once an engine surpasses 3500, its win probability gap against the human record of 2882 is already close to 100%. The historical peak of the top human super grandmaster is only 2882 points, and this vast gap reveals the fundamental chasm between an engine's "computational ability" and a human's "comprehension ability."

However, what these engines output is numerical evaluation (e.g., "+2.3" meaning White is better) and sequences of optimal moves—essentially a black box that "knows the optimal solution but cannot explain it in human language." Large language models are exactly the opposite: skilled at wording and describing logic, but with inherent limitations in precise spatial reasoning on the board. The difficulty LLMs have with board reasoning has deep architectural roots—the model is essentially a statistical pattern matcher based on token sequences. In the model's eyes, the board state is just a string of symbols rather than a real network of two-dimensional spatial relationships. When the model tries to reason "whether it's legal for a knight to jump from e5 to f7," it relies on similar patterns in the training data rather than genuine geometric computation. Research shows that LLMs easily produce illegal moves (i.e., "hallucinations") on blindfold chess problems. Even a powerful model like GPT-4 can have an illegal move rate as high as 20–40% in blindfold tests without assistive tools. How to combine the two is precisely the central proposition raised by Stephan Steinfurt, an engineer at Munich-based company TNG in Germany, in his talk.

The Digital Language of Chess: FEN and PGN

In the chess programming ecosystem, there are two core standard formats for digitally representing a game, and understanding them helps in understanding how AI systems "read" the board. FEN (Forsyth-Edwards Notation) is a compact single-line string that fully encodes a board snapshot at a given moment, including the piece positions in each rank, the current turn, castling rights, en passant target square, and half-move counter. For example, the starting position is encoded as "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1". PGN (Portable Game Notation) is a complete game record format that records all moves in order using standard algebraic notation, along with metadata tags (both players, date, event, etc.). These two formats constitute the common interface language between chess databases, engine APIs, and AI toolchains—at the low level, the tool set in Steinfurt's system is precisely what validates move legality by parsing FEN states and passes the board to the engine for evaluation.

According to him, a major German newspaper recently discussed various new avenues for combining AI and chess, citing expert William Weber's statement—that making AI explain chess like a human coach is "the Holy Grail of chess programming," and predicting it might take another five years to achieve this goal. Yet the results demonstrated by Steinfurt's team suggest this goal may arrive sooner than expected.

What they built is a YouTube chess channel entirely operated automatically by AI—every night it automatically downloads games, analyzes them, generates explanatory videos, and uploads them, all without human intervention. The channel has now accumulated about 500,000 views and over 4,000 subscribers, with most subscriptions concentrated within the past month, showing clear growth momentum.

Data Source: Lichess and the Open Game Ecosystem

The data source for the system's "automatic nightly game downloads" is precisely open platforms like Lichess. Lichess was founded by French developer Thibault Duplessis in 2010 and currently has over 150 million registered accounts, producing about 500,000 game records per day. All of its game data is publicly released in PGN format on database.lichess.org, with monthly data packages reaching tens of GB in size, making it the most important public training corpus in the field of chess AI research—the Maia engine's 100M+ training samples came from here. This data accessibility enables individual developers and small teams to build real human behavior models, giving rise to an open research ecosystem based on Lichess, without relying on closed proprietary datasets. Although Chess.com has over 100 million monthly active users, its game data is not fully open, forming a stark contrast in data openness strategy between the two.

Actual Results: How AI Explains a Brilliant Move Trap

In the demonstration, Steinfurt played a two-minute explanatory video automatically generated by AI, using a complex middlegame as an example: Black's queen is attacked by White's rook on the H-file and tries to move to G4 seeking safety, only to step into a carefully designed trap set by White.

White plays "Rook takes H5," actively sacrificing the piece. Whether Black recaptures with a pawn or the queen, White can follow up with "Knight to F6, check"—this "octopus knight" simultaneously attacks the king on G8 and forks the queen, forming a fatal double fork. The video describes this "dazzling sacrifice" in vivid language and emphasizes: "When the king is exposed, beware of these jumping pieces."

Interestingly, this video was automatically generated in the early hours of that day. Steinfurt admitted that he had originally prepared another video, but when checking in the morning he found that this newly generated version explained the position more brilliantly, so he swapped it in on the spot. This detail perfectly illustrates the actual productivity and content quality of the automated pipeline.

The Technical Core: A Combination of Agent + Tool Set

How exactly does this system work? Steinfurt points out that the key to the entire solution lies in building an AI Agent equipped with a large number of tools, and choosing the right underlying LLM.

An AI Agent is an autonomous system architecture with an LLM as its core decision-making engine, equipped with the ability to call external tools. Its working mode usually follows the ReAct (Reasoning + Acting) framework—this framework was published by Yao et al. at NeurIPS in 2022, and its key contribution was proving that interleaving natural language reasoning traces with action steps can significantly outperform using Chain-of-Thought alone or action sequences alone. At the implementation level, ReAct guides the model through specific prompt templates to produce a looping structure of "Thought → Action → Observation," where the return value of each tool call serves as new context input, enabling the model to correct its reasoning direction based on external feedback. In the chess explanation scenario, this means that when the engine evaluation tool returns "Black -3.2," the model will not continue along its original analysis path, but re-reason "why is this so unfavorable" and decide to call the check-enumeration tool for further confirmation—each step is dynamically decided based on the result of the previous step, rather than following a pre-fixed process. In recent years, ReAct has evolved into variants such as Parallel ReAct, which supports parallel tool calls, and the Reflexion framework, which introduces reflection steps, further improving the quality of completing complex tasks. Compared with a traditional Python script pipeline, the core advantage of the Agent architecture lies in its flexibility—it can dynamically decide which tools to call and how many times based on the specific situation of the game. This flexibility is beyond the reach of a fixed script pipeline, and makes it closer to the analytical thinking of a human coach.

The Standardized Foundation of Tool Calling: Function Calling and the MCP Protocol

The Agent's ability to call "tools" such as chess engines and legal-move validators relies on the structured tool-calling interface of modern LLMs. The Function Calling mechanism introduced by OpenAI in 2023 allows developers to declare tool signatures in JSON Schema format. During reasoning, the model can output structured tool-call requests instead of free text, and the platform is responsible for routing execution and returning results to the model's context. The Model Context Protocol (MCP) proposed by Anthropic in 2024 further standardizes this mechanism into a cross-platform protocol, allowing tool servers to be deployed independently and reused by multiple AI clients, gradually becoming an industry standard. For a chess system, this means that the Stockfish engine, Maia predictor, and board manipulation library can each be encapsulated as independent MCP services, which the Agent combines and calls on demand, with the tool provider and the model provider fully decoupled—this modular architecture is precisely the technical foundation of the system's scalability.

Regarding model selection, he offered several valuable hands-on judgments:

• Gemini 3 Pro (recently released) is the best-performing model he has seen on chess tasks. From the reasoning traces, one can clearly see that its understanding of chess far exceeds previous versions, and the team speculates that Google did deep chess-specific post-training.
• Before this, the best-performing model was "surprisingly" Grok 4, and other models like OpenAI's also have decent basic chess knowledge.

It's worth noting that most of the top models mentioned here are Reasoning Models—such models produce a relatively long "internal chain of thought" (Chain of Thought) before generating the final answer, improving accuracy on complex problems through self-reflection and step-by-step verification. In the chess explanation scenario, reasoning models can simulate a player's analytical process: considering multiple variation lines, weighing the pros and cons of different candidate moves, rather than jumping straight to conclusions, making the generated explanatory content more logically layered.

Around the LLM, the team built a complete set of dedicated tools:

Legal Move Tool

Preventing the Agent from "fabricating" moves that are entirely illegal on the board is a basic but necessary safety guardrail. This tool exists precisely to address the core flaw of LLMs—because large language models work through token prediction rather than truly "understanding" the board's spatial state, they are extremely prone to generating move descriptions that violate chess rules without external constraints, and a single illegal move renders the entire explanation untrustworthy. Enforcing validation through the tool effectively eliminates such hallucinations, which is why the team listed this tool as the top priority.

Board Manipulation Tool

Gives the Agent a complete board, letting it autonomously make moves, take back moves, explore variations, and call the chess engine at any time to evaluate the position.

Check/Capture/Threat Tool

This was originally a classic training method for chess beginners—first focus on "which are the checks, captures, and threats." Steinfurt particularly emphasized that this tool not only provides the best move but also lists all candidate moves, some of which are obviously bad. This kind of "contradictory information" is actually beneficial: humans often consider these moves, and the AI needs to explain why they are bad, rather than only focusing on the single optimal solution deemed best by the engine.

Web Search Tool

Used to supplement the historical background of a game, such as who has played a certain move before and what classic commentary exists.

Architecture Evolution: From Script-Driven to Reasoning-Model-Driven

Steinfurt reviewed the evolution of the project, revealing a key architectural shift.

When the project started, the team used Python scripts to analyze games and aggregate information (check moves, engine evaluations, etc.), then fed the structured results to an LLM to generate descriptions—the thinking was mainly done by scripts. This approach was essentially "tool-led, LLM-polished," with the LLM being merely the final-step copy generator.

As reasoning models matured, the situation changed fundamentally: the Agent can autonomously think about the game, proactively calling tools and assembling knowledge at appropriate points, making the entire system more flexible and closer to a human coach's analytical approach. The center of power in the architecture shifted from script logic to the model's reasoning ability itself.

He also drew on the University of Toronto's Maia engine (referred to as the Maya engine in the talk)—a highly innovative research project fundamentally different from traditional engines like Stockfish that pursue the optimal solution. Maia was jointly developed by the University of Toronto, Microsoft Research, and other institutions, with the paper published at the 2020 KDD conference, representing an important paradigm shift in computer chess research: its goal is not to maximize the probability of winning, but to maximize the accuracy of predicting the moves human players actually make. The research team collected over 100 million real games from the Lichess platform and trained separate neural network models for different ELO brackets (1100 to 1900). Its accuracy in predicting the moves of 1500-bracket players is about 52%, far exceeding Stockfish's roughly 26%.

Behind this project lies a profound insight from cognitive science: the essence of human learning is "error correction" rather than "answer presentation." The 17-ply-deep optimal move given by traditional engines often exceeds the comprehension of ordinary players, and more importantly, cannot tell them "what mistake they are most likely to make." By reverse-engineering human decision patterns, Maia reveals the systematic cognitive biases of players at different skill levels—weaker players tend to ignore the opponent's threats, while stronger players tend to overly pursue material advantage—establishing a true "human behavior predictor." This methodology has also been ported to other educational fields, where researchers have tried using similar ideas to model students' typical error paths in mathematical proofs. Introducing Maia into Steinfurt's system essentially introduces a layer of "empathy filter": although not perfect, it provides valuable input for "explaining mistakes humans easily make." The system pursues not absolute optimality, but a fusion of the best move and the most human-like move.

Production Pipeline and Real Costs

After completing the analysis, the system converts the results into a special format and then renders them into a video. Specific stages include:

• Speech Synthesis: Using ElevenLabs v3, currently a leading platform in the field of speech synthesis. TTS technology has developed over decades, from early rule-based concatenative synthesis, to HMM statistical parametric synthesis, to the current deep-learning-based end-to-end neural network synthesis, with audio quality now approaching human-level. The core breakthrough of ElevenLabs v3 lies in incorporating emotion control into the generation process: the system is typically based on a variational autoencoder (VAE) or diffusion model, encoding text semantics, speaker identity, and emotional state as separate latent vectors, then collaboratively synthesizing the audio waveform during decoding. The technical difficulty of emotion control lies in the multi-dimensional acoustic representation of "emotion"—the speaking rate, fundamental frequency contour (F0 contour), energy envelope, and pause distribution must vary in coordination to produce a natural emotional perception. ElevenLabs v3 introduces a fine-grained prosody control interface, allowing emotions (such as "excited") to be annotated directly in the text, and the system automatically converts these intents into corresponding acoustic parameter combinations, making the explanation sound passionate rather than mechanically read. Research shows that instructional speech with appropriate emotional fluctuations has an information retention rate about 40% higher than monotone reading—a difference with substantial impact in automated content production.
• Visual Presentation: The Agent itself decides which squares to highlight, which arrows to draw, and whether a move should be marked as a "brilliant move."

Regarding costs, Steinfurt gave candid figures: a short video costs about 30 cents, while a longer video may reach several euros. The team currently does not deliberately compress costs, preferring to "err on the side of being too detailed in descriptions," but also admits there is room for optimization—for example, the Agent sometimes repeatedly analyzes the same game, producing redundant tool calls.

In terms of reliability, the error rate is actually quite low—about one "strange description" appears every 20 videos, such as missing a checkmate. Even so, the video usually still contains sufficient valuable information. His mindset is also shifting: from initially insisting on manually reviewing every video, he gradually moved toward "even if there are occasional problem videos, just take them down afterward."

Value and Outlook: Producing Explanatory Content at Scale for Ordinary Players

Why do this? Steinfurt's answer points directly at a market gap. Well-known streamers like Gotham Chess won't specifically explain ordinary people's games. And the input of this system is precisely real human games—in theory, it could generate explanatory videos for your own games to share with friends and family.

He emphasized that the team's philosophy is to pursue the quality of chess content itself, rather than piling on gimmicks like an "exploding king" to chase views. At the same time, they are exploring tiered content for audiences of different levels: for strong players, a video about a "one-move checkmate" is meaningless, but for true beginners, it's exactly what's needed to explain why it's a checkmate. The "human behavior modeling" approach revealed by the Maia engine provides technical support for this—by predicting the typical mistake patterns of players at different ELO levels (systematic cognitive biases such as weaker players ignoring threats and stronger players overly pursuing material advantage), the system can specifically design explanatory content of varying difficulty, rather than imposing top-engine moves on ordinary players. This balance is still being explored.

The Platform Compliance Boundaries of Automated Channels

A fully automated YouTube channel is technically feasible but faces a gray area of platform policy. YouTube's Reused Content Policy explicitly prohibits mass uploads of others' content or auto-generated content without substantial original value, and channels may face demonetization or even bans. The reason Steinfurt's system can operate within this policy framework is that its content has "original added value": the AI explanation layer is natural-language content originally generated by the model on top of the engine analysis, rather than simply reposting game recordings. Chess games themselves are not protected by copyright in most jurisdictions (move sequences are considered facts rather than creative expression), further reducing the copyright risk of the content source. However, the fact that the channel has not yet reached the monetization threshold also shows that even with technical compliance, the algorithmic distribution logic's support for vertical niche content remains very limited, and the commercialization of automated content still needs to overcome structural obstacles in the platform ecosystem.

It's worth mentioning that the channel is not yet profitable, having not reached YouTube's monetization threshold, and is currently running at a net loss. This precisely represents the real situation of the large number of AI content automation projects today—the technical feasibility has been verified, but the commercial loop remains to be closed.

The value of this project may not lie in whether it makes money immediately, but in demonstrating a clear path: using an AI Agent to coordinate a professional engine and an LLM, translating the machine's precise computation into knowledge understandable by humans, and achieving fully automated, scalable production. Starting from chess explanation, this paradigm of "professional tool + reasoning model + automated production" can be fully transferred to more vertical fields requiring professional interpretation—whether it's Go game review, financial analysis, or medical image interpretation, the core logic is the same: let professional tools skilled at precise computation and LLMs skilled at natural language expression work together through an Agent architecture, bridging the chasm between "knowing the optimal solution" and "explaining why."

Key Takeaways