Pre-Window Curvature
A curvature measure for quantifying the influence of historical states on current decisions in reinforcement learning. It is defined as the proportion by which states more than m steps in the past reduce the current marginal gain, and is used to control the length of the history window on which a policy depends.
Timeline (last 90 days)
The paper introduces pre-window curvature c_m, which quantifies the degree to which states visited more than m steps earlier reduce the current marginal gain.
A smaller c_m indicates weaker historical influence, allowing the policy to be more myopic; a larger c_m requires retaining a longer history window.
For a fixed m, the algorithm achieves an approximation ratio of (1-c_m)/(2-c_m) (ignoring estimation error), and this factor approaches 1/2 as c_m decreases.
Estimation error in c_m affects practical performance, and the algorithm assumes decoupled dynamics—how to extend it to scenarios with coupled agent state transitions remains an open problem.
All Facts (4)
The paper introduces pre-window curvature c_m, which quantifies the degree to which states visited more than m steps earlier reduce the current marginal gain.
50%UnverifiedA smaller c_m indicates weaker historical influence, allowing the policy to be more myopic; a larger c_m requires retaining a longer history window.
50%UnverifiedFor a fixed m, the algorithm achieves an approximation ratio of (1-c_m)/(2-c_m) (ignoring estimation error), and this factor approaches 1/2 as c_m decreases.
50%UnverifiedEstimation error in c_m affects practical performance, and the algorithm assumes decoupled dynamics—how to extend it to scenarios with coupled agent state transitions remains an open problem.
50%