Llama 3
The third-generation Llama open-source large language model released by Meta. Its training data includes multilingual corpora, and the model shows notable improvements in multilingual understanding and reasoning capabilities compared to its predecessors.
Core Facts
The model lineup currently supported by Cursor includes the OpenAI GPT-4o series, Anthropic Claude 3.x series, Google Gemini series, and some open-source models.
When Meta previously released LLaMA 2 and LLaMA 3, it used a custom community license that set additional authorization thresholds for companies with more than 700 million monthly active users.
Ollama is an open-source local large language model runtime framework that supports open-source models such as Llama 3, Mistral, CodeLlama, and DeepSeek Coder.
Timeline (last 90 days)
Meta stated that Llama 3 switched to a 128K token vocabulary, using up to 15% fewer tokens than Llama 2 on the same text.
The @cf/meta/llama-3-8b-instruct used in the article is the 8-billion-parameter instruction-tuned version of the Llama 3 series released by Meta.
Llama 3 is an open-weight model family released by Meta, with common specifications of 8B and 70B; open weights mean that after downloading, inference can be done completely offline and it can be fine-tuned yourself.
Open-source models such as the Llama 3 series that adopt similar chunked compression or sparsification techniques more or less have similar phase sensitivity problems.
Grouped-query attention (GQA) was proposed by Ainslie et al. in 2023 and is a compromise between multi-head attention (MHA) and multi-query attention (MQA); Llama 3.1 8B uses 32 query heads divided into 8 groups, with each group of 4 query heads sharing 1 KV head, shrinking the KV cache by 4x compared to MHA.
Meta used more than 16,000 H100s to train the Llama 3 series, and leading institutions' training clusters generally consist of tens of thousands of GPUs or more.
The Llama 3 series uses special tokens such as <|begin_of_text|>, <|start_header_id|>user<|end_header_id|> to delineate role boundaries.
Multilingual models such as mT5 and Llama 3 explicitly design language budgets when building tokenizers, allocating vocabulary slots by writing system or language family.
Small open-source models such as the 8B or 70B parameter versions of the Llama 3 series, after fine-tuning, can be self-hosted on local GPU servers with extremely low marginal cost.
Meta introduced code names such as Scout and Maverick in Llama 3.
28 more timeline events
All Facts (20)
The model lineup currently supported by Cursor includes the OpenAI GPT-4o series, Anthropic Claude 3.x series, Google Gemini series, and some open-source models.
75%VerifiedWhen Meta previously released LLaMA 2 and LLaMA 3, it used a custom community license that set additional authorization thresholds for companies with more than 700 million monthly active users.
75%VerifiedOllama is an open-source local large language model runtime framework that supports open-source models such as Llama 3, Mistral, CodeLlama, and DeepSeek Coder.
70%Unverified可用于本地部署的模型从轻量级的Phi-3、Gemma到重量级的Llama 3、Qwen2.5都可以根据硬件条件自由切换
90%UnverifiedLlama 3、Qwen、DeepSeek等开源模型部署到本地后几乎零边际成本
85%UnverifiedLlama 3 8B模型（80亿参数）运行约需7GB显存
80%Unverifiednpcpy的模型微调使用Beta为0.1的DPO，并用SFT迁移了Llama 3的学术风格用于科学写作场景
80%Unverified通义千问系列在中文响应速度上明显优于Llama 3，因为其预训练语料库包含大量高质量中文文本且采用了针对中文优化的分词器
75%UnverifiedLlama 3提供了8B和70B两个版本，在英文通用任务上的表现已经接近GPT-4级别的闭源模型
70%UnverifiedMeta's Llama 3 made extensive use of synthetic data generated by Llama 2 during training to enhance specific capabilities.
70%UnverifiedLlama 3、Qwen 2.5等开源模型在代码生成任务上已接近商业模型水平
70%UnverifiedMeta's Llama 3 training used more than 16,000 H100 GPUs.
70%Partially VerifiedAnythingLLM支持通过Ollama运行Llama 3、Mistral、Qwen等开源模型，也可接入OpenAI GPT-4、Claude、Gemini等商业模型API
65%UnverifiedMainstream open-source models such as Llama 3 and Qwen2/3 have already adopted GQA.
60%UnverifiedLocal inference is constrained by the user's hardware: a 7B-parameter quantized model (such as the Q4 version of Llama 3 8B) has a file size of approximately 4–5 GB, can reach 20–40 tokens per second on a Mac with an Apple M-series chip, and may drop to 3–5 tokens per second on a laptop with only an integrated GPU and 8 GB of RAM.
60%UnverifiedLocally running open-source LLM models typically have a context capacity of only 4K to 32K tokens, limited by hardware memory.
60%UnverifiedMeta stated that Llama 3 switched to a 128K token vocabulary, using up to 15% fewer tokens than Llama 2 on the same text.
50%UnverifiedThe @cf/meta/llama-3-8b-instruct used in the article is the 8-billion-parameter instruction-tuned version of the Llama 3 series released by Meta.
50%UnverifiedLlama 3 is an open-weight model family released by Meta, with common specifications of 8B and 70B; open weights mean that after downloading, inference can be done completely offline and it can be fine-tuned yourself.
50%UnverifiedOpen-source models such as the Llama 3 series that adopt similar chunked compression or sparsification techniques more or less have similar phase sensitivity problems.
50%