Llama 3.3 70B
Llama 3.3 70B is an open-source large language model released by Meta, belonging to the third generation of the Llama series with approximately 70 billion parameters. The model supports a range of natural language processing tasks including text generation, dialogue, reasoning, and code writing. Released in open-source form, it can be deployed locally or in the cloud by researchers and developers, offering high inference efficiency and performance.
Core Facts
Using float16 precision as a baseline, each parameter occupies approximately 2 bytes, meaning a 7B model requires approximately 14GB of VRAM/memory, and a 70B model requires approximately 140GB.
Even after quantization, 70B-level models often require more than 40 GB of VRAM or a multi-GPU configuration.
A 300B INT4 quantized model requires approximately 150–180GB of storage space.
Timeline (last 90 days)
Llama 3.3 70B scored 93.4% on one hosting provider and only 69.5% on another.
MainBrella's dialogue reasoning is handled by Llama 3.3 70B, invoked through Cloudflare Workers AI.
Cloudflare Workers AI is billed by neurons or call volume and offers out-of-the-box open-source models such as Llama 3.3 70B, invoked via API, but model selection, fine-tuning and context window are limited by the platform.
Fine-tuning a 70B model requires about 50 GB of VRAM for QLoRA, about 150 GB for LoRA, and up to 1.1 TB for full fine-tuning with the AdamW optimizer.
Every 1,000 in-flight tokens require about 0.33 GB of VRAM in the KV cache.
The KV cache calculation formula for Llama 3.3 70B is: 2 (keys and values) × 80 layers × 8 KV heads × 128 head dimension × 2 bytes = 327,680 bytes per token.
Full fine-tuning of a 70B model with the AdamW optimizer requires about 1,129 GB of VRAM (70.55 billion parameters × 16 bytes per parameter).
Running a 4-bit quantized 70B model on 1×A100 80GB costs about $1.09 per hour and can support about 98,000 in-flight tokens.
Running a 16-bit precision 70B model on 2×80GB GPUs leaves only 2.9 GB of VRAM after the weights are loaded, accommodating only about 9,000 in-flight tokens, which in practice cannot serve requests effectively.
DataBugBench testing covers five models: Claude 3.5 Sonnet, GPT-4o, Gemini 1.5 Pro, Qwen 2.5 Coder 32B, and Llama 3.3 70B.
6 more timeline events
All Facts (20)
Using float16 precision as a baseline, each parameter occupies approximately 2 bytes, meaning a 7B model requires approximately 14GB of VRAM/memory, and a 70B model requires approximately 140GB.
80%VerifiedEven after quantization, 70B-level models often require more than 40 GB of VRAM or a multi-GPU configuration.
75%VerifiedA 300B INT4 quantized model requires approximately 150–180GB of storage space.
65%VerifiedAfter 4-bit quantization, Llama 3.3 70B has a model size of approximately 40 GB and can run on a Mac Studio with 64 GB of memory or a single professional GPU with 48 GB of VRAM.
65%UnverifiedLlama 3.3 70B支持函数调用（Function Calling）功能
90%UnverifiedLlama 3.3 70B is essentially a fine-tuned version of the earlier Llama 3.1 70B, trained using supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF).
60%UnverifiedLlama 3.3 70B scored 93.4% on one hosting provider and only 69.5% on another.
50%UnverifiedCloudflare Workers AI is billed by neurons or call volume and offers out-of-the-box open-source models such as Llama 3.3 70B, invoked via API, but model selection, fine-tuning and context window are limited by the platform.
50%UnverifiedMainBrella's dialogue reasoning is handled by Llama 3.3 70B, invoked through Cloudflare Workers AI.
50%UnverifiedEvery 1,000 in-flight tokens require about 0.33 GB of VRAM in the KV cache.
50%UnverifiedRunning a 16-bit precision 70B model on 2×80GB GPUs leaves only 2.9 GB of VRAM after the weights are loaded, accommodating only about 9,000 in-flight tokens, which in practice cannot serve requests effectively.
50%UnverifiedFull fine-tuning of a 70B model with the AdamW optimizer requires about 1,129 GB of VRAM (70.55 billion parameters × 16 bytes per parameter).
50%UnverifiedRunning a 4-bit quantized 70B model on 1×A100 80GB costs about $1.09 per hour and can support about 98,000 in-flight tokens.
50%UnverifiedFine-tuning a 70B model requires about 50 GB of VRAM for QLoRA, about 150 GB for LoRA, and up to 1.1 TB for full fine-tuning with the AdamW optimizer.
50%UnverifiedThe KV cache calculation formula for Llama 3.3 70B is: 2 (keys and values) × 80 layers × 8 KV heads × 128 head dimension × 2 bytes = 327,680 bytes per token.
50%UnverifiedDataBugBench testing covers five models: Claude 3.5 Sonnet, GPT-4o, Gemini 1.5 Pro, Qwen 2.5 Coder 32B, and Llama 3.3 70B.
50%UnverifiedLlama 3.3 70B's kinship term selection accuracy was 77.92%, while its generation accuracy was only 24.24%.
50%UnverifiedUnder conditions where the target language is explicitly specified in the prompt (L3 prompts), GLM-5.1 achieved a generation accuracy of 72.29%, while Llama-3.3 70B reached only 24.24%.
50%UnverifiedAfter adopting GQA, Llama 3.3 70B has a KV Cache of approximately 320 KB per token.
50%UnverifiedThe research team conducted an example-masking experiment on Llama 3.3 70B, in which accuracy dropped after masking the example content and recovered to above the zero-shot baseline after the examples were restored.
50%