CartPole
A classic control task in reinforcement learning whose goal is to balance a pole so that it does not fall. The state space is extremely simple and episodes are short, giving the task very low complexity.
Timeline (last 90 days)
CartPole is the most classic introductory reinforcement learning environment in OpenAI Gym, with a 4-dimensional state and 2 actions, and typically a few hundred lines of code and a few dozen seconds are enough to train a stable policy.
In the controlled experiment by De Oliveira et al., PPO won on all tasks except CartPole; GRPO's group weighting mechanism did not yield better performance than a naive batch mean.
The five classic control tasks usually refer to CartPole, Acrobot, MountainCar, Pendulum, etc., whose common feature is that the state changes continuously, the reward is per-step or approximately sparse, and the required precision of policy timing is far higher than LLM generation.
PPO may not have the advantage on CartPole because CartPole has low state dimensionality, short episodes, and extremely low task difficulty, and the additional overhead and estimation error brought by the critic may drag down performance, so CartPole is the exception but not representative.
Applying VQC to classic RL benchmark environments such as CartPole and FrozenLake has been covered in multiple papers available for reproduction and comparison.
CartPole问题最早可追溯到1960年代的控制理论研究，是验证控制算法稳定性的经典测试平台
CartPole是验证RL算法正确性的首选环境，若DQN实现无法在CartPole上收敛则在更复杂环境上基本没有希望
CartPole最早由Barto、Sutton和Anderson在1983年提出，作为自适应控制的基准问题
All Facts (8)
CartPole is the most classic introductory reinforcement learning environment in OpenAI Gym, with a 4-dimensional state and 2 actions, and typically a few hundred lines of code and a few dozen seconds are enough to train a stable policy.
50%UnverifiedIn the controlled experiment by De Oliveira et al., PPO won on all tasks except CartPole; GRPO's group weighting mechanism did not yield better performance than a naive batch mean.
50%UnverifiedThe five classic control tasks usually refer to CartPole, Acrobot, MountainCar, Pendulum, etc., whose common feature is that the state changes continuously, the reward is per-step or approximately sparse, and the required precision of policy timing is far higher than LLM generation.
50%UnverifiedPPO may not have the advantage on CartPole because CartPole has low state dimensionality, short episodes, and extremely low task difficulty, and the additional overhead and estimation error brought by the critic may drag down performance, so CartPole is the exception but not representative.
50%UnverifiedApplying VQC to classic RL benchmark environments such as CartPole and FrozenLake has been covered in multiple papers available for reproduction and comparison.
50%UnverifiedCartPole问题最早可追溯到1960年代的控制理论研究，是验证控制算法稳定性的经典测试平台
50%UnverifiedCartPole是验证RL算法正确性的首选环境，若DQN实现无法在CartPole上收敛则在更复杂环境上基本没有希望
50%UnverifiedCartPole最早由Barto、Sutton和Anderson在1983年提出，作为自适应控制的基准问题
50%