MB-RS-QVI
A model-based risk-sensitive reinforcement learning algorithm that first estimates the MDP transition model from samples, then performs risk-sensitive Q-value iteration on the estimated model. It is a plug-in method designed for policy learning under recursive entropic risk preferences.
Timeline (last 90 days)
The paper does not propose an entirely new algorithm, but instead conducts a more refined theoretical analysis of the existing model-based risk-sensitive Q-value iteration method MB-RS-QVI.
MB-RS-QVI is a plug-in method that first estimates the MDP transition model from collected samples and then performs risk-sensitive Q-value iteration on the estimated model.
The new sample complexity bound matches the known lower bound in its exponential dependence on |β|/(1-γ).
The new bound matches the lower bound up to logarithmic factors in its dependence on the number of states S, the number of actions A, the accuracy ε, and the risk parameter |β|.
The refined analysis in this paper proves that the actual exponential dependence of MB-RS-QVI takes the form e^(O(|β|/(1-γ))), matching the information-theoretic lower bound.
All Facts (5)
The paper does not propose an entirely new algorithm, but instead conducts a more refined theoretical analysis of the existing model-based risk-sensitive Q-value iteration method MB-RS-QVI.
50%UnverifiedMB-RS-QVI is a plug-in method that first estimates the MDP transition model from collected samples and then performs risk-sensitive Q-value iteration on the estimated model.
50%UnverifiedThe new sample complexity bound matches the known lower bound in its exponential dependence on |β|/(1-γ).
50%UnverifiedThe new bound matches the lower bound up to logarithmic factors in its dependence on the number of states S, the number of actions A, the accuracy ε, and the risk parameter |β|.
50%UnverifiedThe refined analysis in this paper proves that the actual exponential dependence of MB-RS-QVI takes the form e^(O(|β|/(1-γ))), matching the information-theoretic lower bound.
50%