When Is the Muon Optimizer Optimal? Radial Fragility and a Minimax Geometric Analysis

The Muon optimizer has performed impressively in neural network training in recent years, replacing the traditional matrix gradient with an approximate polar factor. But why is this update rule effective? Previously, the Isotropic Curvature Model (ICM) offered an explanation from the perspective of curvature: when gradient orthogonalization minimizes a one-step local loss surrogate, Muon's update direction is justified. However, ICM assumes a fixed input radius, an assumption that does not hold in real training. A recent study introduces random radii and anisotropic analysis, revealing the fragility of this explanation and providing a new minimax variational characterization of Muon's polar direction.

From Fixed Radius to Random Radius: Extending ICM

The core idea of ICM is that along the gradient direction, the local loss can be approximated by an isotropic curvature model. When the input radius is fixed, gradient orthogonalization (i.e., Muon's polar factor update) minimizes this surrogate loss. But in real training, the input activation radius of each layer varies with the data distribution and is not constant.

To address this, the researchers propose the Random-Radius Isotropic Curvature Model (RR-ICM), replacing the fixed radius with a general radial distribution. They also define the Polar-Factor Admissibility Set, which precisely encodes the geometric conditions for "scaled polar optimality"—that is, under which gradient forms the Muon update is optimal.

Polar factor originates from the polar decomposition of a matrix: any real matrix $G$ can be uniquely written as $G = UH$, where $U$ is a column-orthogonal (or isometric) matrix and $H$ is a positive-semidefinite symmetric matrix. $U$ is the polar factor of $G$, obtainable from the singular value decomposition $G = U\Sigma V^\top$ as $U = UV^\top$. Geometrically, the polar factor uniformly scales all singular values of $G$ to 1, retaining only its "direction" information, hence it is also called the orthogonalization of the matrix gradient. The Muon optimizer uses exactly this operation in place of the raw matrix gradient update in traditional SGD/Adam. Understanding the polar factor is key to understanding the entire article: the so-called "scaled polar optimality" discusses—under which gradient forms does replacing the gradient with its polar factor minimize the local loss surrogate.

Core Finding: Zero-Capacity Rigidity under Atomless Radial Distributions

By deriving the exact characterization of the admissibility set under arbitrary radial distributions, the researchers discovered a key result: all non-scalar admissible responses arise from radial atoms aligned with curvature inflection points. In other words, if the radial distribution is atomless (i.e., continuous), then exact scaled polar optimality can hold only when the gradient itself is already a scaled column isometry.

This is called zero-capacity rigidity. It means that under common continuous radial distributions, ICM's average curvature explanation can hardly provide non-trivial support for Muon's update—Muon's optimality becomes extremely rare. This result directly exposes the radial fragility of explanations based on average curvature: as soon as the radius is no longer a fixed constant, the original explanation quickly fails.

Scaled column isometry means the column vectors of matrix $G$ are not only mutually orthogonal but also have equal norms (or norms scaled by the same scalar), i.e., $G^\top G = c,I$ ($c>0$). For such matrices, the polar factor differs from the matrix itself only by a positive scalar—that is, it is "almost already" a polar factor. This is precisely the meaning of zero-capacity rigidity: when the radial distribution is continuous (atomless), the only gradient form for which the Muon update is exactly optimal is such a matrix that has already been "well normalized." In other words, for general gradients, Muon's optimality no longer holds under continuous radial distributions. Here, "atom" refers to discrete probability masses in the radial distribution, corresponding to special radii at curvature inflection points; only responses at these points can produce non-trivial optimality.

Minimax Perspective: Connecting Spectral Geometry and the Polar Direction

To bridge the gap between the curvature framework and Muon's known spectral geometry, the researchers further propose Minimax RR-ICM. Unlike averaging over directions, this model optimizes over probability distributions on the sphere in a worst-case manner.

They prove that every non-zero global minimizer lies on the polar ray of the gradient, and its scale is determined by a one-dimensional convex problem. The corresponding admissible geometry is a positive-semidefinite trace slab, in sharp contrast to the spherical zonotope geometry of RR-ICM.

This minimax variational characterization provides a new theoretical basis for Muon's polar direction—it no longer relies on fragile average curvature assumptions, but robustly derives the polar direction through worst-case analysis.

Zonotope and positive-semidefinite trace slab are two different convex geometric objects. A zonotope is generated by the Minkowski sum of several line segments; in RR-ICM it describes the spherically symmetric geometry of the admissibility set. A positive-semidefinite trace slab refers to the set of positive-semidefinite matrices of the form ${X \succeq 0,\ \mathrm{tr}(X) = c}$, constraining the matrix's eigenvalues to be non-negative with fixed trace. Polar ray refers to the ray starting from the origin with direction given by the gradient's polar factor direction $U$, i.e., ${tU : t>0}$. The minimax result states that under worst-case optimization, the global minimizer must lie on this ray, with the scale $t$ determined by a one-dimensional convex problem. Spectral geometry refers to geometric properties related to the distribution of matrix singular values (the spectrum), serving as the background framework for understanding the relationship between Muon updates and singular value structure.

Implications for Optimizer Design

This work conveys two signals. First, Muon's effectiveness may not be simply attributable to gradient orthogonalization under average curvature, because that explanation is extremely unstable under general radial distributions. Second, the minimax framework provides a more robust geometric foundation for understanding Muon, linking the polar direction to positive-semidefinite trace slab geometry.

For practitioners, this means that Muon's excellent performance may stem from its worst-case optimization properties rather than average-case curvature matching. Future optimizer design can draw on this minimax perspective to maintain theoretical interpretability across a broader range of radial distributions.

One-line Summary

Research reveals that the average curvature explanation of the Muon optimizer fails under random radii and proposes a new minimax characterization.

Paragraph Summary

This article analyzes the theoretical question of why the Muon optimizer is effective. The previous Isotropic Curvature Model (ICM) explained from a curvature perspective: when the input radius is fixed, gradient orthogonalization minimizes the local loss surrogate. However, in real training the input radius varies with data, prompting researchers to propose Random-Radius ICM (RR-ICM) and define a polar-factor admissibility set that precisely characterizes scaled polar optimality conditions. The core finding is zero-capacity rigidity: under atomless continuous radial distributions, exact scaled polar optimality holds only when the gradient itself is already a scaled column isometry, meaning the original average curvature explanation almost fails in general cases. To further bridge the gap with spectral geometry, the researchers propose Minimax RR-ICM, proving that non-zero global minimizers lie on the gradient's polar ray, corresponding to positive-semidefinite trace slab geometry, providing a robust variational characterization of Muon's polar direction that does not rely on fragile average curvature assumptions.