Ape-X
Ape-X is a distributed reinforcement learning architecture proposed by DeepMind. It uses a large number of Actors to collect experiences in parallel, writing them into a shared prioritized experience replay buffer, while a single centralized Learner performs parameter updates. By fully abandoning synchronous gradients, Ape-X inherently avoids parameter overwrite conflicts.
Timeline (last 90 days)
Architectures such as IMPALA and Ape-X are representative implementations of the sampling-training decoupling pattern, achieving throughput improvements of tens of times compared to single-machine synchronous training on tasks such as Atari.
Actor-Learner decoupling is a common approach in distributed reinforcement learning, and architectures such as A3C, IMPALA, and Ape-X are all based on this concept.
Ape-X completely abandons synchronous gradients and relies on a shared prioritized experience replay pool to decouple collection from learning.
All Facts (3)
Architectures such as IMPALA and Ape-X are representative implementations of the sampling-training decoupling pattern, achieving throughput improvements of tens of times compared to single-machine synchronous training on tasks such as Atari.
50%UnverifiedActor-Learner decoupling is a common approach in distributed reinforcement learning, and architectures such as A3C, IMPALA, and Ape-X are all based on this concept.
50%UnverifiedApe-X completely abandons synchronous gradients and relies on a shared prioritized experience replay pool to decouple collection from learning.
50%