Actor-Critic
Actor-Critic is a class of algorithm frameworks in reinforcement learning that combines the advantages of policy-based and value-based methods. The Actor selects actions according to the current policy, while the Critic evaluates the value of those actions and generates an error signal to guide policy updates. This framework effectively reduces the variance of policy gradient methods and improves training stability. It is widely applied in domains such as robotic control, game intelligence, and natural language processing.
Timeline (last 90 days)
Dreamer's policy learning adopts the Actor-Critic framework: the Actor outputs an action distribution based on the imagined latent state s_t, and the Critic estimates the value of that state; gradients are backpropagated through the world model's latent trajectory, but the world model parameters are usually frozen to avoid the policy optimization signal contaminating the world model's learning objective, thereby maintaining training stability.