Unverified50% confidenceFactExact time
Ripple 65B uses a dense architecture rather than a mixture-of-experts model, on the grounds that dense models are simpler to train and infer, and it reports reaching about 40% MFU in pretraining and post-training without custom kernels or quantization.
1
Sources
50%
Confidence
Long-term
Relevance
11/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCompared to MoE architectures, dense models have more uniform gradient flow during training, are less prone to expert collapse, and exhibit more deterministic inference behavior.70% similarUnverified监督训练的ResNet-50内在维度相对CLIP更高,说明对比预训练产生了更紧凑的语义流形70% similarUnverifiedByteDance's Seed team plans to train a foundation large model exceeding 5 trillion parameters, currently in the early validation stage, rejecting the distillation route and insisting on native pre-training.67% similarUnverifiedA systematic empirical study used nearly 200B tokens for pretraining on decoder-only LLMs ranging from 360M to 3.1B parameters, examining fourteen orthogonal training variants.67% similarUnverifiedIn the future, the staged training idea can be extended to more complex hybrid architecture designs, such as dynamically adjusting the update pace of different layers or introducing more fine-grained attribution analysis to guide training strategies.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1166491API
curl https://kongchang.com/api/v1/knowledge/claims/1166491MCP
get_claim(id=1166491)