Training PPO on a MacBook: A Browser-Based Cabo Card Game with Hidden Information

Training a Hidden-Information Card Game AI from Scratch: Self-play PPO Meets Cabo

In the reinforcement learning community, hidden-information card games are regarded as challenges closer to real-world decision-making than board games due to their imperfect-information nature. Recently, a developer shared an intriguing project on Reddit: based on the rules of Cabo (a popular hidden-information card game), they trained an AI from scratch using the Self-play PPO algorithm, completed entirely on a MacBook, and provided a demo that can be played directly in the browser.

The core value of this project lies in lowering the practical barrier to multi-agent reinforcement learning. No GPU cluster is needed, no massive distributed sampling—a consumer-grade laptop can run the full training loop and let the model play against human players in real time.

Why Is Cabo an Ideal Testbed for Reinforcement Learning?

Cabo (similar to memory flip-and-swap mechanics) has simple rules but a complex information structure: each player can only see part of their own hand, the rest is hidden, and players must infer opponents' states through drawing, swapping, and memory. This partial observability naturally suits reinforcement learning for strategy exploration.

The developer chose a variant of PPO (Proximal Policy Optimization) for self-play, letting the model continuously compete against historical versions of itself, thereby learning card-playing strategies without supervised signals. Compared to supervised learning that relies on human annotations, self-play can continuously drive strategy evolution—a standard approach for complex game tasks since AlphaZero.

It's worth noting that Cabo's randomness (the draw pile) imposes higher demands on strategy stability. PPO's clipped objective limits abrupt policy changes during updates, and combined with the curriculum effect of self-play, the model gradually transitions from random play to decisions involving memory and reasoning.

Partially Observable Markov Decision Process (POMDP) is the standard mathematical framework for describing hidden-information games. Unlike fully observable MDPs, in a POMDP the agent cannot directly know the true state of the environment and can only infer it indirectly through observations. In Cabo, players cannot see opponents' hands or the deck order; they can only form a belief state based on revealed swap actions and their own hand. Policy gradient methods like PPO under POMDPs typically need to incorporate memory mechanisms (such as RNNs or Transformers) to integrate historical observations; otherwise the policy degrades due to missing information. This also explains why Cabo places higher demands on network architecture design—the model must learn to extract hidden information from action sequences.

Running Training on a MacBook: Key Engineering Trade-offs

According to the project share, training was completed entirely on a MacBook, implying the team made fine-grained trade-offs between algorithmic efficiency and hardware constraints. Typically, hidden-information games require longer training cycles to converge, and consumer-grade hardware has limited memory and compute, meaning network architecture, parallel environments, and experience replay mechanisms all need lightweight design.

Browser playability further lowers the experience barrier: the model may be exported via ONNX or a similar format for local inference in the web frontend, avoiding server dependency. This "train—export—frontend play" loop provides a reusable paradigm for other enthusiasts. For developers wanting to get started with multi-agent reinforcement learning, this project offers a directly runnable template rather than staying stuck in paper formulas.

PPO's clipped objective avoids policy collapse from excessive update steps by restricting the probability ratio between new and old policies to the interval [1-ε, 1+ε]. In self-play scenarios, this mechanism is especially important: opponent policies change continuously during training, and if updates are too aggressive, the model easily overfits the current opponent and loses generalization ability. Combined with Generalized Advantage Estimation (GAE), PPO balances bias and variance, suitable for turn-based, sparse-reward games like Cabo. Engineering-wise, lightweight design typically includes reducing hidden layer dimensions, using multi-environment parallel sampling (e.g., vectorized environments), and shortening rollout length to maintain stable sample throughput under laptop memory constraints.

Play Instantly in the Browser: The Last Mile from Training to Deployment

The project deploys the model on the browser side, so users can play without installing any environment. Behind this, the PyTorch model typically needs to be converted to a lightweight inference format and executed in the frontend with a JavaScript engine. For turn-based card games like Cabo, inference latency requirements are low, and frontend inference is entirely feasible.

This deployment approach has a hidden benefit: protecting user privacy, as all games are computed locally. At the same time, it demonstrates another possibility for deploying reinforcement learning models—not necessarily to the cloud; edge devices can also host policy inference.

ONNX (Open Neural Network Exchange) is a cross-framework model exchange format that supports exporting PyTorch-trained models as framework-agnostic computation graphs. After export, inference can be executed in the browser via ONNX Runtime Web using WebAssembly or WebGPU, without a Python backend. For turn-based card games like Cabo, a single inference involves only a small number of matrix operations, with frontend latency typically in milliseconds—fully meeting real-time play needs. This edge inference approach also avoids network transmission overhead and server costs, allowing the demo to be statically hosted and further lowering the sharing barrier.

Implications for AI Application Developers

The most direct inspiration from this project is that cost-controllable reinforcement learning experiments are no longer out of reach. Many developers still associate RL with "requiring massive compute," but the combination of Self-play PPO + consumer-grade hardware shows that for games with clear rules and moderate state spaces, individual developers can complete training and validation locally.

Second, hidden-information games provide a rich testbed for AI research. Unlike fully observable games, these require models to make decisions under uncertainty, closer to real-world recommendation, negotiation, or resource scheduling scenarios. If the training experience on Cabo can be transferred to more complex real-world problems, the value would be even greater.

Finally, browser playability means results can be quickly validated. In open-source communities, demos that can be played directly often receive more feedback than projects that only release code, which in turn helps developers iterate.

Limitations and Future Directions

The project is still a personal experiment; the original post did not disclose quantitative metrics such as training duration, model size, or win rate. Cabo's randomness also means strategy evaluation requires many games to be stable. If training curves and benchmarks could be published in the future, it would be more conducive to community reproduction and improvement. Additionally, self-play strategies may fall into local optima, requiring diversity mechanisms or opponent pools to improve robustness—these are necessary steps for the project to move from demo to reliable AI.