Unverified50% confidenceFactExact time
Full fine-tuning of a 70B model with the AdamW optimizer requires about 1,129 GB of VRAM (70.55 billion parameters × 16 bytes per parameter).
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen using the Adam optimizer, 16 bytes of state information are required per parameter (4 bytes for FP32 parameters + 4 bytes for FP32 gradients + 4 bytes for first-order momentum + 4 bytes for second-order momentum), requiring at least approximately 24 GB of state information per GPU for a 1.5 billion parameter model.80% similarVerifiedA 70B-parameter FP16 model requires approximately 140 GB of video memory, approximately 70 GB after INT8 quantization, and approximately 35–40 GB after INT4 quantization.77% similarUnverifiedFine-tuning a 70B model requires about 50 GB of VRAM for QLoRA, about 150 GB for LoRA, and up to 1.1 TB for full fine-tuning with the AdamW optimizer.76% similarUnverifiedRunning an open-source model at the 70-billion-parameter scale requires at least approximately 140GB of VRAM in FP16 precision, i.e., 8 H100s in parallel.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1019929API
curl https://kongchang.com/api/v1/knowledge/claims/1019929MCP
get_claim(id=1019929)