Server Momentum
A weighted history mechanism for aggregating updates in federated learning, typically implemented as an exponential moving average, used to smooth the training trajectory and reduce oscillations caused by client data heterogeneity.
Timeline (last 90 days)
Server momentum is a weighted-history mechanism for aggregating updates, usually implemented as an exponential moving average (EMA): after each round of aggregation, the server does not directly adopt the average gradient of the current round but instead mixes it with the model parameters of the previous round at a certain weight.
For the encoder role, RASO retains server momentum to stabilize the encoder's adaptation process, even when certain modalities are missing on some clients.
All Facts (2)
Server momentum is a weighted-history mechanism for aggregating updates, usually implemented as an exponential moving average (EMA): after each round of aggregation, the server does not directly adopt the average gradient of the current round but instead mixes it with the model parameters of the previous round at a certain weight.
50%UnverifiedFor the encoder role, RASO retains server momentum to stabilize the encoder's adaptation process, even when certain modalities are missing on some clients.
50%