Cubic Doggo

After training is complete, the model is exported as an ONNX file for deployment; the authors added a TRAIN_CONFIG field to the ONNX file to record all training configurations for easy traceability.

The Cubic Doggo project introduced random horizontal pushing forces during training, applying random forces of up to 5N to the 3 kg robot to simulate lateral impacts in real environments.

Jitter during training in the GUI originates from parameter exploration and is normal; but implementation correctness must first be tested with zero actions, especially when a large number of link indices is involved.

The project's logging system recorded termination reasons, showing that 20% of terminations were due to pitch angle exceeding 45 degrees, caused by left and right lateral pushing forces.

The project saves log data as .csv files so they can be handed directly to an LLM to analyze trends; this approach of using an LLM to assist in analyzing training curves offers a new idea for reinforcement learning debugging.

Proposed solutions to the four-footed grounding problem include adding foot contact rewards, adjusting torque penalty weights, incorporating symmetry constraints, and improving the anti-slip mechanism.

The Cubic Doggo project code has been open-sourced on GitHub (SphericalCowww/CubicDoggo_06Z).

The Cubic Doggo project uses the PPO algorithm from Stable Baselines3, together with the Gymnasium environment interface and the MuJoCo physics engine, to train a homemade robot dog to stand.

The robot dog's action space contains 12 joint values (4 of which are mimic passive mirror joints) and 12 control quantities, with a single episode limit set at 20 seconds.

The developer suspects that penalty_joint_torque (joint load penalty) is the culprit behind the two-legged standing problem, as the optimizer tends to reduce the number of weight-bearing legs.

reward_height is a Gaussian reward centered on the body being approximately 15 cm above the ground, and reward_roll_pitch is a Gaussian reward centered on both roll and pitch being 0.

penalty_joint_pos_init gradually reduces its weight as training progresses through curriculum learning, and the penalty_slip anti-slip penalty has not yet been implemented.

The Cubic Doggo project is open-sourced on GitHub and includes configuration files, training code, and deployment code.

In the training results, the PPO optimizer converged to an unintended posture of supporting the body on only two legs, rather than stable four-footed grounding.