Llama 3.1 405B
Llama 3.1 405B is a large-scale open-source language model released by Meta AI, with 405 billion parameters, making it the largest model in the Llama series. It supports multilingual text understanding and generation, complex reasoning, code writing, and long-context processing. Model weights are made available to researchers and developers for academic research, commercial applications, and fine-tuning, with performance on several benchmarks comparable to leading closed-source large models.
Core Facts
Timeline (last 90 days)
The Ascent GX10 supports dual-unit stacking expansion, supporting development and inference of models with up to 405B parameters.
A single GX10's native memory capacity is usually insufficient to independently host a full 405B model; the claimed maximum support for 405B relies on the aggregated memory pool after dual-unit stacking, along with quantization and tensor parallelism techniques.
By the end of 2024, open-source models such as Llama 3.1 405B and Qwen2.5-72B had approached or in some benchmarks even surpassed GPT-4 level performance.
Llama 3.1 405B's score on MMLU is already close to the level of GPT-4o, and Qwen 2.5 72B has surpassed GPT-4 Turbo on certain metrics in math and code generation benchmarks.
Meta used more than 16,000 H100 GPUs to train Llama 3.1 405B, with training taking several months and costs potentially exceeding $100 million.
All Facts (6)
Meta's released Llama 3.1 405B has approached GPT-4 levels across multiple benchmark tests.
80%VerifiedMeta used more than 16,000 H100 GPUs to train Llama 3.1 405B, with training taking several months and costs potentially exceeding $100 million.
65%UnverifiedLlama 3.1 405B's score on MMLU is already close to the level of GPT-4o, and Qwen 2.5 72B has surpassed GPT-4 Turbo on certain metrics in math and code generation benchmarks.
60%UnverifiedThe Ascent GX10 supports dual-unit stacking expansion, supporting development and inference of models with up to 405B parameters.
50%UnverifiedA single GX10's native memory capacity is usually insufficient to independently host a full 405B model; the claimed maximum support for 405B relies on the aggregated memory pool after dual-unit stacking, along with quantization and tensor parallelism techniques.
50%UnverifiedBy the end of 2024, open-source models such as Llama 3.1 405B and Qwen2.5-72B had approached or in some benchmarks even surpassed GPT-4 level performance.
50%