DP-Muon
A differentially private version of the Muon optimizer, based on matrix orthogonalization updates with added privacy-preserving mechanisms.
Timeline (last 90 days)
The optimizers covered by DP-μP include DP-SGD, DP-Adam and DP-Muon.
DP-Adam is a privacy version that embeds the clipping-and-noising step of DP-SGD into the momentum estimation of the Adam optimizer; DP-Muon is a privacy version that uses the Muon optimizer (based on matrix orthogonalization updates) for matrix parameters.
All Facts (2)
The optimizers covered by DP-μP include DP-SGD, DP-Adam and DP-Muon.
50%UnverifiedDP-Adam is a privacy version that embeds the clipping-and-noising step of DP-SGD into the momentum estimation of the Adam optimizer; DP-Muon is a privacy version that uses the Muon optimizer (based on matrix orthogonalization updates) for matrix parameters.
50%