Non-stationarity
A core challenge in multi-agent reinforcement learning referring to the phenomenon whereby the continuous updating of multiple agents' policies causes the environmental distribution faced by each agent to shift constantly, invalidating convergence guarantees that rely on a fixed-environment assumption.
Timeline (last 90 days)
Interaction-induced non-stationarity violates the stationarity assumption of MDPs, invalidating convergence guarantees derived from that assumption.
The PACT method attempts to directly model and adapt to interaction-induced non-stationarity, enabling agents to explicitly identify and adapt to distributional shifts caused by interactions with other agents.
The real world is non-stationary, and a policy's performance during training may deteriorate sharply after the environment undergoes a distribution shift.
Stock price series are typical non-stationary data, and the substantial change between the data distribution at model training time and at prediction time is referred to in statistics as 'concept drift'.
All Facts (4)
Interaction-induced non-stationarity violates the stationarity assumption of MDPs, invalidating convergence guarantees derived from that assumption.
50%UnverifiedThe PACT method attempts to directly model and adapt to interaction-induced non-stationarity, enabling agents to explicitly identify and adapt to distributional shifts caused by interactions with other agents.
50%UnverifiedThe real world is non-stationary, and a policy's performance during training may deteriorate sharply after the environment undergoes a distribution shift.
50%UnverifiedStock price series are typical non-stationary data, and the substantial change between the data distribution at model training time and at prediction time is referred to in statistics as 'concept drift'.
50%