Adversarial MDP
A reinforcement learning setting in which the environment may choose an arbitrary loss function at each round, possibly depending on the algorithm's history, rather than sampling from a fixed distribution. Feedback is bandit-style, requiring the algorithm to be more robust to uncertainty.
Timeline (last 90 days)
A recent theoretical study, by regularizing the Q-function, improves the regret bound of policy optimization under adversarial MDPs by a full factor of H, thereby eliminating the horizon gap.
In adversarial MDPs, the environment can choose an arbitrary loss function at each episode (possibly depending on the algorithm's history), and the loss can vary arbitrarily over time, so standard optimistic or confidence upper bound methods no longer apply directly.
All Facts (2)
A recent theoretical study, by regularizing the Q-function, improves the regret bound of policy optimization under adversarial MDPs by a full factor of H, thereby eliminating the horizon gap.
50%UnverifiedIn adversarial MDPs, the environment can choose an arbitrary loss function at each episode (possibly depending on the algorithm's history), and the loss can vary arbitrarily over time, so standard optimistic or confidence upper bound methods no longer apply directly.
50%