Browser Game + Reinforcement Learning: Players Build Robots and Watch Neural Networks Learn to Walk

Title

Browser Game + Reinforcement Learning: Players Build Robots and Watch Neural Networks Learn to Walk

Meta Description

Developer creates browser game Bitbots: players assemble robots, train them to walk with reinforcement learning, and watch policy network activity in real time through a "brain view." The game is built on MuJoCo simulation with GPU training via the MJX Warp backend and is currently in closed testing.

Article Content

A developer shared his new project Bitbots on Reddit: a browser game where players assemble robots from parts and then train them to walk through reinforcement learning. What's more interesting is that the game offers a "brain view" that displays the policy network's inputs, hidden layer activations, and motor outputs in real time, letting players intuitively see how activity inside the controller changes as the robot moves.

The game is currently in closed beta, based on MuJoCo simulation and using GPU training via the MJX Warp backend. Players decide the robot's body and joint configuration, while the policy network learns how to control it. During matches, the robots act autonomously. The developer also poses an open question: with a limited training budget, how can you support wildly creative robot designs without the reward function pushing everyone toward the same way of walking?

More Than Just Watching It Walk: Exposing the Policy Network to Players

Most reinforcement learning demos only tell you the result—the robot walks, or it falls over. Bitbots wants to do a bit more: visualize the internal activity of the controller. The "brain view" in the video shows the policy network's inputs, selected hidden layer activations, and motor outputs, synchronized with the robot's actual movement. The developer emphasizes that these are recorded values from the real policy, not decorative animations.

The point of this design is that players can observe the connection between network activity and movement. Although it doesn't explain what any single neuron "means," it at least gives players a window into the controller's internals. For a game, this transparency reduces the black-box feel of reinforcement learning and makes debugging more traceable—if the robot walks awkwardly, you can at least see what the network is "thinking."

Tech Stack: MuJoCo + MJX Warp Backend

Bitbots' simulation relies on MuJoCo, and training is accelerated on the GPU through the MJX Warp backend. Players choose the robot's body and joint configuration, and the policy network learns to control it. During the competition phase, the robot acts fully autonomously, and players cannot intervene mid-run.

This tech combination is common in robot reinforcement learning research, but bringing it into a browser game and letting players participate in morphology design is an interesting experiment. MuJoCo's physical accuracy plus GPU training means that even if players design strange joint structures, the simulation can still provide relatively credible dynamic feedback. However, real-time training and simulation on the browser side still have performance requirements, which is one reason the developer chose the MJX Warp backend.

MuJoCo (Multi-Joint dynamics with Contact) is a physics simulation engine originally developed at the University of Washington, later acquired by DeepMind and open-sourced. It is known for high-precision contact dynamics and computational efficiency, and is commonly used in robot control, biomechanics, and reinforcement learning research. MJX is the JAX implementation of MuJoCo, supporting the parallel execution of large numbers of simulation environments on the GPU, thereby accelerating reinforcement learning training. Warp is a high-performance computing framework introduced by NVIDIA for accelerating physics simulation and graphics tasks. MJX's Warp backend combines JAX's automatic differentiation with Warp's GPU kernels, further improving simulation speed and making real-time training in the browser possible.

Core Challenge: Creative Morphologies vs. Limited Training Budget

The developer himself raises a key question: with a limited training budget, how can you support creative, non-traditional robot designs? He hopes players can experiment with all kinds of strange body and joint configurations, rather than being steered toward a few "known feasible" designs.

Behind this is a classic tension in reinforcement learning locomotion research. If the reward function is designed too specifically, all robots will converge to similar gaits; if it is too loose, training may not converge at all, or may learn strange behaviors that exploit loopholes. For robots with different morphologies, the same reward function is hard to generalize. The developer wants to know how others working on multi-morphology locomotion would handle this problem.

One possible approach is for the reward function to define only the "goal" (such as forward speed and stability) while leaving the "method" for the policy to discover on its own. But even so, learning difficulty varies enormously across morphologies, and allocating the training budget remains tricky. Another direction is curriculum learning or morphology randomization, letting the policy see more diverse bodies during training, but this further increases training costs.

Reward function design is crucial in reinforcement learning, as it defines the goals the agent should pursue. For multi-morphology locomotion, a common challenge is that the reward function may be too specific, causing robots with different morphologies to converge to similar gaits (for example, all bipedal robots learning to take tiny steps), or too loose, leading to unstable training. Curriculum Learning is a training strategy that improves learning efficiency by gradually increasing task difficulty, for example starting from simple terrain and gradually introducing complex terrain. Morphology Randomization randomly changes the robot's body structure during training so that the policy can generalize to multiple morphologies, but it increases training costs. These methods are often used to address morphology diversity issues.

Closed Testing and How to Participate

Bitbots is currently in closed alpha. The developer says free invitations are available through comments or direct messages, and players can try designing their own robots. The project page is at https://bitbots.endresgames.com .

For players interested in reinforcement learning, robot morphology design, or gamified AI training, this is an experiment worth watching. It turns things usually hidden in papers and laboratories—policy networks, the training process, and the coupling between morphology and control—into something you can play with. As for the question the developer raised, the answer may only become clear after more players design strange robots.

One-line Summary

Browser game Bitbots lets players assemble robots and train them to walk through reinforcement learning, offering a "brain view" that shows activity inside the policy network.

Paragraph Summary

Bitbots is a browser game in closed testing where players can assemble robots and train them to walk through reinforcement learning. The game is based on MuJoCo simulation and GPU training via the MJX Warp backend; players customize the body and joint configuration, and the policy network learns to control movement. Its signature feature is the "brain view," which displays the policy network's inputs, hidden layer activations, and motor outputs in real time to help players understand the controller's internal activity. The developer raises an open question: with a limited training budget, how can creative morphologies be supported without the reward function leading to gait homogenization? Free invitations are currently available through comments or direct messages.