Linear Mixture MDP
An MDP model in reinforcement learning in which the state transition probabilities can be expressed as a mixture of multiple linear functions. It is suitable for function approximation settings, and algorithms in this work also achieve horizon-dependent improvements under this setting.
Timeline (last 90 days)
The paper extends the algorithm to adversarial linear mixture MDPs and obtains the same improvement in horizon dependence, indicating that the regularized Q-function idea does not depend on the finiteness of the tabular setting.
A linear mixture MDP is a class of function approximation settings in which the transition probability is assumed to be a linear combination of known basis features: P(s'|s,a)=Σφ_i(s'|s,a)θ_i, where φ_i are known features and θ are unknown parameters, allowing the state space to be huge or even continuous while retaining a learnable linear structure.
All Facts (2)
The paper extends the algorithm to adversarial linear mixture MDPs and obtains the same improvement in horizon dependence, indicating that the regularized Q-function idea does not depend on the finiteness of the tabular setting.
50%UnverifiedA linear mixture MDP is a class of function approximation settings in which the transition probability is assumed to be a linear combination of known basis features: P(s'|s,a)=Σφ_i(s'|s,a)θ_i, where φ_i are known features and θ are unknown parameters, allowing the state space to be huge or even continuous while retaining a learnable linear structure.
50%