Unverified50% confidenceOpinionExact time
PPO may not have the advantage on CartPole because CartPole has low state dimensionality, short episodes, and extremely low task difficulty, and the additional overhead and estimation error brought by the critic may drag down performance, so CartPole is the exception but not representative.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn PPO and GRPO algorithms, the importance sampling correction weight is proportional to the ratio of new and old policy probabilities exp(logprob_new - logprob_old); due to the amplification of the exponential function, a logprob discrepancy of 0.001 can produce a substantial weight error.57% similarUnverifiedFor problems such as gradual performance degradation or intermittent anomalies that do not manifest as a clear failure of a single operation, Dllog's fail-then-replay model may miss important clues.56% similarUnverifiedPOMDP的转移概率、观测模型、奖励函数参数一旦估计失准，求解出的最优策略可能不如保守的阈值规则可靠，对初学者小项目而言存在过度工程风险55% similarUnverifiedThe paper points out that mainstream MOBO methods have notable shortcomings when the evaluation budget is extremely tight, potentially failing to generate enough solutions to simultaneously approximate the entire front.55% similarUnverifiedThe limitation of PPO is its relatively low sample efficiency, typically requiring millions to billions of game frames to reach human-level performance.55% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1025866API
curl https://kongchang.com/api/v1/knowledge/claims/1025866MCP
get_claim(id=1025866)