Ling 3.0 Flash
A hybrid MoE large language model released by InclusionAI. It adopts a grouped hybrid attention architecture combining KDA and MLA, with the decoder composing lightweight and global attention layers in a five-to-one ratio.
Timeline (last 90 days)
InclusionAI's Ling 3.0 Flash adopts a hybrid MoE design, in which the decoder groups five Kimi Delta Attention (KDA) layers with one Multi-head Latent Attention (MLA) layer, and is natively supported in Transformers with the loading class BailingHybridForCausalLM.
vLLM v0.28.0 adds support for models including Muse Glimmer, Ling 3.0 Flash, Dots3 NOTE, and Interns2mobius.
All Facts (2)
InclusionAI's Ling 3.0 Flash adopts a hybrid MoE design, in which the decoder groups five Kimi Delta Attention (KDA) layers with one Multi-head Latent Attention (MLA) layer, and is natively supported in Transformers with the loading class BailingHybridForCausalLM.
50%UnverifiedvLLM v0.28.0 adds support for models including Muse Glimmer, Ling 3.0 Flash, Dots3 NOTE, and Interns2mobius.
50%