Training a PPO Agent for the Hidden-Information Card Game Cabo on a MacBook

Training a Card AI That Can "Read Minds" from Scratch

Recently, a developer shared his project on Reddit: training an agent for the hidden-information card game Cabo using self-play PPO, with the entire training process completed on a MacBook. Cabo is a memory-based card game where players cannot see their own cards. Each player has 4 face-down cards, and at the start they may peek at only 2 of them. The goal is to end the round with the lowest possible total value in hand. Card abilities allow you to peek at your own cards, scout an opponent's card, or perform a blind swap.

You can play directly against this agent at lowpeek.xyz. It runs entirely client-side in the browser, with a response time of about 3 milliseconds per move (according to the developer).

Lightweight Architecture and Training Configuration

The agent uses the PPO algorithm with GAE (γ=0.999, λ=0.95), an action space of 43 masked actions, and an observation dimension of 292. Observations include: your own and your opponent's card slots (along with what you know about them), discard pile counts, scores, and memory for each slot (the source of each card, how long it has been there, and what the opponent last discarded).

A key auxiliary design is the belief head: it predicts the value of each hidden card and is trained with cross-entropy loss against the true card faces. This auxiliary task lets the agent learn some interesting statistical regularities, such as "cards your opponent deliberately swaps into your hand average about 10.7 points, while blindly swapped cards average about 6.4 points."

Training also uses potential-based reward shaping to optimize score differences, along with a small penalty for wasted peeks. The opponent pool mixes self-play, historical snapshots, frozen old networks, hand-written rule bots, and random policies.

PPO (Proximal Policy Optimization) is a policy-gradient-based reinforcement learning algorithm that ensures training stability by limiting the magnitude of each policy update. It uses a clipped surrogate objective to prevent the policy from changing too much in a single update. GAE (Generalized Advantage Estimation) is a method for estimating the advantage function that trades off bias and variance through the parameters γ (discount factor) and λ (smoothing factor). A γ close to 1 (such as 0.999) means the agent focuses more on long-term returns, while λ=0.95 applies an exponentially weighted average to advantage estimates to reduce variance. In card games, GAE can effectively handle sparse rewards and delayed returns.

Action masking is a key technique for handling variable action spaces. In Cabo, not all 43 actions are legal at any given time (for example, you cannot use a swap card when you do not have the corresponding card in hand). By setting the probabilities of illegal actions to zero and renormalizing, the agent samples only from legal actions, avoiding invalid exploration. The 292-dimensional observation encodes all visible information and memory features, enabling the network to make decisions based on the complete state.

Slot Symmetry: An Easily Overlooked Detail

The 4 slots in a Cabo hand are symmetric, giving 4!·4! = 576 total permutations. During training, the developer used random permutations, but at play time averaged the policy outputs over 24 fixed permutations. Without this treatment, the agent might make wrong decisions depending on which slot a card occupies—for example, swapping away a 13 when it should have swapped away a 9.

This detail reveals a general challenge in hidden-information games: policies need to be robust to state symmetries, otherwise the learned policy overfits to specific slot indices.

In hidden-information card games, state symmetry refers to states that are physically different but strategically equivalent. In Cabo, for example, the 4 slots in a hand are indistinguishable under the rules, but the neural network sees an input vector with slot indices. If only one fixed slot permutation is used during training, the network may bind certain strategies to specific slot indices rather than learning the true relationships between card values. This is a form of overfitting that causes the agent's performance to degrade when slot indices change.

Common ways to address symmetry include: data augmentation (randomly shuffling slot order during training), parameter sharing (designing network architectures equivariant to permutations), or inference-time ensembling (averaging outputs over multiple symmetric transformations). The developer adopted a simple approach: random permutations during training and averaging over 24 fixed permutations at play time. This amounts to test-time ensembling and improves policy robustness at low cost. Similar issues exist in games such as Texas Hold'em and mahjong, and they are a detail that must be considered when designing AI for hidden-information games.

Achieving High-Speed Training on a MacBook

To make training feasible on a laptop, the entire game was vectorized into tensors and run on Apple's GPU (MPS). The developer reports about 1.5M environment steps per second across 131k tables, and about 200k learning steps per second when the network is in the loop. All logic was verified move by move against a reference Python implementation.

However, MPS has two pitfalls: tensors do not support bitwise shift operations, and argsort handles ties differently from the CPU, so shuffling is performed on the CPU.

MPS (Metal Performance Shaders) is Apple's GPU acceleration framework for its own chips (M1/M2/M3, etc.), and PyTorch supports tensor computation on Mac through the mps backend. Compared with CUDA, the MPS ecosystem is newer, and some operations are not yet fully supported or behave inconsistently. For example, bitwise shift operations may not be implemented on MPS tensors, and the sorting stability of argsort when encountering equal elements may differ from the CPU, which affects operations that depend on random permutations, such as shuffling.

The developer performs shuffling on the CPU precisely to avoid these inconsistencies. In reinforcement learning training, environment steps and learning steps are two key metrics for measuring throughput. Environment steps refer to the number of interactions between the agent and the environment, while learning steps refer to the number of network parameter updates. Reaching 1.5M environment steps per second across 131k tables means the vectorized environment fully utilizes the GPU's parallel capability; 200k learning steps per second reflects the efficiency of network forward and backward propagation. Throughput at this scale makes large-scale self-play possible on consumer-grade hardware.

Training Results and Emergent Behavior

Starting from random weights, the agent reached a 51% win rate against the rule bot after 1 minute and 90% after 5.5 hours. The current version achieves a 91.8% win rate over 4,000 games. However, the head-to-head win rate between consecutive versions is about 50%, indicating that performance is plateauing.

The agent spontaneously learned some surprising strategies:

• Deliberately losing a round to reset the score: When it holds a hand with a known total of 22 and its total score is 73, it deliberately knocks to end the round, knowing it will lose that round. Because 73+22+5 penalty = exactly 100, triggering a reset to 50. Confidence: 100%.
• Precise card swapping: When using a swap card, it dumps a known high card onto the opponent (87% of the time), and if it has scouted an opponent's low card, it targets that card precisely (88%).
• Deliberate pairing: It will actively take a worse card to form a pair, so that it can clear both cards with a single multi-swap in the next round.

Emergent behavior refers to complex strategies that an agent spontaneously develops during training and that are not explicitly specified in the reward function. In reinforcement learning, when an agent faces rich environment dynamics and has sufficient exploration, it may discover optimization strategies that human designers did not anticipate. The Cabo agent's learned behavior of "deliberately losing a round to reset the score" is a typical example: it exploits the game rule that a total score of 100 resets to 50, and by precisely calculating its current hand and penalty points, it actively chooses to lose the current round in exchange for a long-term advantage. This behavior shows that the agent has learned not only short-term scoring but also the game's long-term state transitions.

Similarly, precise card swapping and deliberate pairing also reflect the ability to model opponent beliefs and deck distributions. The emergence of these strategies depends on the hidden-card predictions provided by the belief head and on sufficient training time. However, emergent behavior may also include undesirable risky actions, such as risk exposure in blind multi-swaps. Analyzing these behaviors helps understand the agent's decision logic and guides improvements in reward shaping and network design.

Approaches Tried but That Did Not Work

The developer candidly shared several failed paths:

• Lookahead search over deterministic rollouts: This helped only when the value head was replaced with a win-rate lookup table (because discounted values are biased) and the policy was overridden only when significance was sufficient (z > 2).
• Expert iteration on search labels: This performed worse (48% vs. baseline) because there were too few labels and too much noise.

The agent still takes risks in blind multi-swaps with unknown cards (exposing two cards when it fails), and occasionally calls with a known low pair on the first turn.

Next Steps and Open Questions

The developer plans to anonymously store every game played by humans on the website (seed + moves, with server-side replay verification) in order to evaluate the agent's performance against humans and fine-tune it. He is particularly interested in advice on how to break through the plateau: should he add search at play time, adopt population training, or design a smarter belief-state representation?

This project demonstrates the feasibility of training complex card-game AI on consumer-grade hardware and provides a reproducible lightweight baseline for research on hidden-information games.