batch_size
The number of samples used in each training iteration in machine learning. It affects the variance of gradient estimates and the stability of training.
Timeline (last 90 days)
When batch_size scales linearly with the number of environments, the variance of the gradient estimate decreases at a rate of about 1/batch_size.
With a fixed learning_rate, when batch_size scales linearly with the number of environments and num_epoch remains unchanged, the total displacement pushing the policy toward a new distribution in a single iteration may increase, manifesting as violent oscillation or even collapse of rewards in the early stages of training.
When the number of parallel environments increases from N to M, batch_size should increase at least linearly by a ratio of M/N; buffer_size should remain 5 to 10 times batch_size.
All Facts (3)
When batch_size scales linearly with the number of environments, the variance of the gradient estimate decreases at a rate of about 1/batch_size.
50%UnverifiedWith a fixed learning_rate, when batch_size scales linearly with the number of environments and num_epoch remains unchanged, the total displacement pushing the policy toward a new distribution in a single iteration may increase, manifesting as violent oscillation or even collapse of rewards in the early stages of training.
50%UnverifiedWhen the number of parallel environments increases from N to M, batch_size should increase at least linearly by a ratio of M/N; buffer_size should remain 5 to 10 times batch_size.
50%