Small Neural Network vs. General Decision Model: A Card Game AI Failure Mode Retrospective

A developer who works primarily with Drupal and web development spent a year building a browser-based Hearthstone-style card game from scratch, using it as a learning project to understand game AI mechanics. Rather than relying on established machine learning frameworks, he deliberately kept dependencies minimal — hand-implementing everything from state representation and legal action generation to MCTS search and neural network evaluation. That process alone was impressively hardcore, but the genuinely valuable insights emerged when he put his small neural network and a general-purpose decision model called Jev at the same table. Their failure modes turned out to be far more revealing than the final score.

A 248k-Parameter Neural MCTS Agent

The core of this system is a fully connected action-value network with the architecture 1857 → 128 → 64 → 32 → 16 → 1, using ReLU activations in the hidden layers and a linear output layer — approximately 248,705 trainable parameters. It estimates Q(s,a) values for state-action pairs.

Of the 1,857 inputs, 1,842 are state features and 15 are action features. The state portion includes general game features, a hero identity vector, two 768-dimensional board embeddings, and a hashed hand representation. Notably, the embedding generator is separate from this 248k-parameter scorer — meaning the network has an extraordinarily rich board representation, but only 15 explicit features describing the candidate action itself. This asymmetry later becomes key to explaining the network's behavioral biases.

The training strategy also evolved over time. The author initially had the network play against a hand-written heuristic board evaluator, providing a relatively stable learning target and a quick path to a usable baseline. Once the agent stopped acting randomly and produced reasonable value estimates, he switched to self-play, letting the network learn from games generated against its own previous versions rather than continuing to imitate the heuristic rules. That transition was what interested him most: moving from "can the network approximate my hand-written evaluator" toward "can the network improve itself through self-play."

Monte Carlo Tree Search (MCTS) is a sampling-based decision algorithm widely used in board and card game AI. Its core loop has four steps: selection (traversing down from the root node using some policy), expansion (adding new child nodes at leaf nodes), simulation (running a random or heuristic rollout from the new node to a terminal state), and backpropagation (updating statistics along the path with the simulation result). When a neural network is involved, it typically plays two roles: replacing expensive random rollouts with a "value head" to estimate win probability, and using a "policy head" to output prior probabilities over actions to guide tree expansion — dramatically improving search efficiency. AlphaGo/AlphaZero are the landmark implementations of this paradigm. The system described here uses only an action-value network (Q-value estimation) rather than a full policy+value dual-head structure, which is the context behind the author's later comparisons between "neural priors" and "uniform priors."

Getting a General Model to the Table Fairly

The other side of the comparison is Jev, a general-purpose decision model exposed via OpenRouter. The core question was simple: can a general model hold its own against a small neural network purpose-built for this specific game?

To make the comparison meaningful, the author refactored the engine so that legal action generation became a canonical module independent of MCTS. Each decision is represented as a finite set of legal actions — playing a card, attacking a specific target, using the hero power, combining card plays with the hero power, or ending the turn. Jev receives only the serialized player-visible game state plus legal action IDs, and must choose one. It makes every real decision itself, including attacks and target selection, with no visibility into the opponent's hidden information. This design ensures both agents face the same level of abstraction.

The matchup was 100 mirror games: identical decks, fixed random seeds, and swapped turn order. The result was nearly a draw — Jev won 48, Neural-MCTS won 52. But what really stands out is that the two agents agreed on their preferred action only about 63% of the time. The disagreements deserve more excavation than the scoreline.

A core engineering challenge when using large language models (LLMs) for game decision-making is structured representation of the action space. A native LLM outputs free text, while a game engine needs precise action IDs; without constraints, the model may hallucinate illegal actions (nonexistent cards, invalid targets) or produce malformed output. The author's solution — enumerating legal actions as a finite set and passing them alongside a serialized state — is the mainstream approach in both academia and industry, sometimes called "constrained decoding" or "action masking." This design also resolves the fairness problem: the neural network takes feature vectors as input while Jev takes a natural-language serialized state, but both face exactly the same legal action set, eliminating confounds like "the neural network is penalized for seeing more illegal actions."

Dissecting Disagreements with a Counterfactual Evaluator

To understand who was right and who was wrong in that 37% of disagreements, the author built a counterfactual evaluator. For each disagreement point, he reconstructed the exact decision state, cloned it twice — applying Jev's action to one copy and the neural policy's action to the other — then ran deeper unguided MCTS from both resulting positions.

The latest round of analysis used 500 disagreement positions, stratified across all 100 games, with 10,000 MCTS iterations per position, a rollout depth of 20, repeated three times, and no neural policy guidance applied. The overall counterfactual results remained close, but the tactical breakdown exposed two starkly different failure modes.

The Neural Network: Systematically Undervaluing Face Pressure

The strongest signal emerged in the tradeoff between "going face" and "developing the board." In 105 positions where Jev chose a face attack while the neural network chose board development, deep MCTS favored Jev in 92 positions, the neural network in only 3, with 10 draws. In a more specific subset — positions where Jev attacked the enemy hero while the neural network chose to play minions — deep MCTS favored Jev 53 to 1 across 63 positions, with 9 draws.

The reverse comparison is equally telling: in 27 positions where Jev attacked enemy minions while the neural network went face, deep MCTS favored the neural network in 25. This means the evaluator isn't simply biased toward Jev in general — the neural policy systematically under-selects face pressure.

The lethal recognition results are even more striking. Among the 500 disagreement positions, 19 contained an immediately executable legal lethal. Jev identified lethal 16 times; the neural network identified it 0 times; in the remaining 3, neither agent found it. These are raw neural recommendations rather than final MCTS-searched actions, but the signal is strong: lethal pressure is simply not salient enough in the network's representation or training objective.

The author's hypothesis is that the network overweights "future board development" relative to "immediate damage potential." It has an extremely rich board representation but only 15 explicit features describing candidate actions. This makes concepts like "more board presence is better" very easy to learn, while relationships like "the opponent is at 6 health, I have 7 attack power available, stop developing and just kill them" can only be inferred indirectly.

Jev: Not Knowing When a Turn Is Actually Over

Jev has nearly the opposite weakness. In 21 positions where Jev chose to end the turn while other legal actions were still available — 16 with playable cards remaining, 5 with legal attacks remaining, and a median of 5 remaining mana — deep MCTS favored the neural network's alternative in all 21 cases.

Independent hero power selection exposed similar sequencing issues. When Jev chose to use the hero power while the neural network chose to develop minions, deep MCTS favored the neural network 29 out of 29 times.

The overall picture becomes unusually clear: the neural network leans toward development and undervalues immediate face damage; Jev is better at tactical aggression but poor at judging when a multi-action turn is truly finished. These two weaknesses are perfectly complementary, which is ultimately why the matchup ended at 48–52.

Counterfactual evaluation draws from causal inference: to measure the true quality of a decision, you need to construct a counterfactual branch from the same initial state — "what would have happened if we'd chosen the other action" — rather than relying solely on the final outcome of the actual game. In game AI, this is typically implemented by forking two independent simulation paths from the disagreement node. The key design choice here is using unguided MCTS (pure Monte Carlo search with a uniform prior and no neural network bias) as the judge, so that the evaluation itself isn't contaminated by either model's strategic biases — if you use a neural network to evaluate its own actions, the conclusions become circularly self-confirming. The cost is enormous computational overhead (thousands of iterations per disagreement point), but the payoff is a relatively neutral third-party perspective that makes "whose action is better under deep search" an operationally answerable question.

An Experiment That Also Exposed the Evaluator's Own Flaws

This experiment also incidentally exposed a bug in the author's own evaluator: the non-terminal heuristic occasionally assigned scores exceeding the value of an actual terminal win. He subsequently adjusted the value scaling so that terminal outcomes strictly dominate — a subtle detail that's easy to overlook but can corrupt the entire training objective.

The planned next steps are methodical: inspect the neural features and training objective before scaling up the network; add more explicit tactical signals if necessary; then compare MCTS performance with uniform priors, neural priors, and Jev-derived priors under the same search budget. The author is particularly interested in root-node-only Jev guidance — having Jev provide a one-time prior over legal actions at the root, with local MCTS handling the actual search from there.

The questions he posed to the community are also thought-provoking: is the missing face signal more of a representation problem, a training-target problem, or both? Should explicit features like isLethal, immediateFaceDamage, and remainingAttack be added, or should the representation stay raw and force the network to infer these relationships on its own?

The Value Beyond the Scoreboard

This project began as a way to self-study how game AI systems work, but the most interesting conclusion is this: when two agents are closely matched in strength, their strategic failure modes carry more information than the scoreboard. A small network with rich representation but sparse explicit action signals will naturally drift toward local optima like "playing minions is safe"; a general-purpose model may have sharp tactical intuitions but exposes weaknesses in the closing sequences of multi-action turns.

For anyone building decision-making systems, the lessons here extend well beyond card games: what a model can learn depends heavily on how much explicit, goal-relevant signal you give it, and whether the training objective lets "the outcomes that actually matter" take center stage. The project code is open-sourced on GitHub (seutje/wow-legends) and playable online — if you want to play against Jev, you'll need to bring your own OpenRouter API key.