Q-learning
One of the most foundational value-function methods in reinforcement learning, proposed by Watkins in 1989. It maintains a Q-table that records the expected cumulative reward for each action in each state, with values updated iteratively via the Bellman equation.
Core Facts
Timeline (last 90 days)
At the algorithmic level, the usage-aware framework can typically be solved with REINFORCE, the reparameterization trick, or value-based Q-learning, with the specific choice depending on whether the experiment space is discrete or continuous and whether the rules are differentiable.
Q-learning's update target is the immediate reward plus the discounted maximum Q value of the next state, so regardless of what behavior policy the agent actually follows, it is approximating the optimal action-value function.
In environments with strong exploration and high risk, Q-learning is more aggressive and SARSA is more conservative, and the two may converge to different policies.
REINFORCE directly computes the gradient of the policy itself without requiring explicit estimation of the value of each state, differing from value-function-based methods such as Q-learning.
In 1989, Chris Watkins proposed the Q-learning algorithm, laying the foundation for model-free reinforcement learning.
The key to DQN lies in combining convolutional neural networks with the Q-learning algorithm, enabling agents to autonomously learn strategies without manually engineered features.
Q-Learning 是 off-policy 的 TD 控制方法，直接学习最优动作价值函数；SARSA 是 on-policy 方法，更新时使用智能体实际选择的下一个动作
Q-Learning is an off-policy method that assumes optimal future actions when updating Q-values; SARSA is an on-policy method that updates Q-values based on the actions actually taken.
All Facts (8)
Q-Learning is an off-policy method that assumes optimal future actions when updating Q-values; SARSA is an on-policy method that updates Q-values based on the actions actually taken.
65%UnverifiedAt the algorithmic level, the usage-aware framework can typically be solved with REINFORCE, the reparameterization trick, or value-based Q-learning, with the specific choice depending on whether the experiment space is discrete or continuous and whether the rules are differentiable.
50%UnverifiedQ-learning's update target is the immediate reward plus the discounted maximum Q value of the next state, so regardless of what behavior policy the agent actually follows, it is approximating the optimal action-value function.
50%UnverifiedIn environments with strong exploration and high risk, Q-learning is more aggressive and SARSA is more conservative, and the two may converge to different policies.
50%UnverifiedREINFORCE directly computes the gradient of the policy itself without requiring explicit estimation of the value of each state, differing from value-function-based methods such as Q-learning.
50%UnverifiedThe key to DQN lies in combining convolutional neural networks with the Q-learning algorithm, enabling agents to autonomously learn strategies without manually engineered features.
50%UnverifiedQ-Learning 是 off-policy 的 TD 控制方法，直接学习最优动作价值函数；SARSA 是 on-policy 方法，更新时使用智能体实际选择的下一个动作
50%UnverifiedIn 1989, Chris Watkins proposed the Q-learning algorithm, laying the foundation for model-free reinforcement learning.
50%