Concept
time_horizon
In reinforcement learning, the number of time steps collected by each environment before a policy update. It affects trajectory length and advantage estimation.
Timeline (last 90 days)
Oct 13
time_horizon cannot be shortened indefinitely; for tasks such as quadruped walking that require multi-step coordinated actions, too short a horizon makes it difficult for the agent to observe a complete gait cycle, and the advantage estimate will be biased due to frequent bootstrapping.
Unverified50%