Concept
LatentMoE
A Mixture-of-Experts architecture variant proposed by NVIDIA in which routing occurs in a low-dimensional latent space rather than the full hidden dimension. This reduces communication and computation overhead, allowing models with larger parameter counts to be deployed under the same memory budget.
Timeline (last 90 days)
Oct 15
Nemotron 3.5 Bolt's architecture adopts LatentMoE mixing Mamba-2 and Transformer, and the vision side uses the C-RADIO v4-H encoder.
Unverified50%