ModelMixtral

Mixtral 8×7B

A sparse Mixture-of-Experts (MoE) language model released by Mistral AI in 2023, comprising eight expert networks of which two are activated per token. It represents a landmark open-source implementation of the MoE architecture for large language models.

Timeline (last 90 days)

Oct 3

Open-source models such as Mixtral 8×7B and DeepSeek-V2 both adopt MoE architecture.

Unverified50%

Sep 15

Switch Transformer was released in 2021, and Mixtral 8×7B was released in 2023.

Unverified50%

All Facts (2)

Unverified

Open-source models such as Mixtral 8×7B and DeepSeek-V2 both adopt MoE architecture.

50%Unverified

Switch Transformer was released in 2021, and Mixtral 8×7B was released in 2023.

50%

Source Articles

Dense Models vs MoE: Active Parameters, Throughput, and a Practical Selection GuideSep 15