Ling-3.0-flash
A lightweight MoE large language model released by Inclusion AI, aimed at efficient inference scenarios.
Timeline (last 90 days)
In NVIDIA's Ling-3.0-flash model, each token selects 8 of 512 routed experts in each MoE layer, plus one shared expert.
Under the batch-size-1 Spark configuration, the cost of a step verifying 4 tokens is roughly equal to 2.26 ordinary decoding steps.
The NVIDIA team adopted a batch size of 1 inference setting when evaluating Inclusion AI's Ling-3.0-flash.
The above conclusions come from a developer blog that uses Ling-3.0-flash as the evaluation target, and cross-validation from multiple sources is still lacking.
All Facts (4)
In NVIDIA's Ling-3.0-flash model, each token selects 8 of 512 routed experts in each MoE layer, plus one shared expert.
50%UnverifiedUnder the batch-size-1 Spark configuration, the cost of a step verifying 4 tokens is roughly equal to 2.26 ordinary decoding steps.
50%UnverifiedThe NVIDIA team adopted a batch size of 1 inference setting when evaluating Inclusion AI's Ling-3.0-flash.
50%UnverifiedThe above conclusions come from a developer blog that uses Ling-3.0-flash as the evaluation target, and cross-validation from multiple sources is still lacking.
50%