OLMo-core 3

An open-source large-scale Mixture-of-Experts (MoE) model training framework developed by Allen AI (AI2). Designed for MoE architectures, it supports scalable training at the trillion-parameter level and provides a fully open training stack including code implementations and engineering optimizations.