Robust Async-Fed-Q
An adversarially robust asynchronous federated Q-learning algorithm that combines variance-reduced Bellman estimation with robust aggregation mechanisms. In multi-agent federated reinforcement learning settings with malicious agents, it preserves the collaborative statistical benefits of honest agents.
Timeline (last 90 days)
A study published on arXiv (arXiv:2610.06918v1) proposes an algorithm named Robust Async-Fed-Q, establishing a theoretical foundation for adversarially robust federated Q-learning.
Robust Async-Fed-Q adopts an epoch-based federated learning framework.
The algorithm employs variance-reduced estimation of the Bellman optimality operator locally at each agent.
The algorithm replaces naive averaging with robust aggregation at the central server to identify and suppress anomalous uploads that deviate from the normal distribution.
The paper proves that the negative impact of adversarial agents diminishes as the amount of data collected by honest agents increases, and the adversarial impact vanishes entirely in the infinite-sample limit.
The paper provides an information-theoretic lower bound and achieves a near-matching upper bound, representing the first nearly matching upper and lower bound result in the field of adversarially robust federated reinforcement learning.
The algorithmic framework is extended to two more realistic sampling scenarios: single-trajectory Markov sampling and heterogeneous partial coverage.
The epoch-based design significantly improves the known optimal communication complexity of federated Q-learning under asynchronous sampling.
All Facts (8)
The epoch-based design significantly improves the known optimal communication complexity of federated Q-learning under asynchronous sampling.
50%UnverifiedRobust Async-Fed-Q adopts an epoch-based federated learning framework.
50%UnverifiedThe algorithm employs variance-reduced estimation of the Bellman optimality operator locally at each agent.
50%UnverifiedThe algorithm replaces naive averaging with robust aggregation at the central server to identify and suppress anomalous uploads that deviate from the normal distribution.
50%UnverifiedThe paper proves that the negative impact of adversarial agents diminishes as the amount of data collected by honest agents increases, and the adversarial impact vanishes entirely in the infinite-sample limit.
50%UnverifiedThe paper provides an information-theoretic lower bound and achieves a near-matching upper bound, representing the first nearly matching upper and lower bound result in the field of adversarially robust federated reinforcement learning.
50%UnverifiedThe algorithmic framework is extended to two more realistic sampling scenarios: single-trajectory Markov sampling and heterogeneous partial coverage.
50%UnverifiedA study published on arXiv (arXiv:2610.06918v1) proposes an algorithm named Robust Async-Fed-Q, establishing a theoretical foundation for adversarially robust federated Q-learning.
50%