Unverified50% confidenceFactExact time
During training, the Adam optimizer additionally stores gradient momentum at approximately 2–3 times the weight size, meaning the full training memory requirement for a 70B model can easily exceed 400 GB.
1
Sources
50%
Confidence
Long-term
Relevance
9/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe model weights alone of a 175-billion-parameter model (in BF16 precision) require 350 GB of storage, and the Adam optimizer must maintain two sets of states—first-order moments and second-order moments—totaling more than 1 TB.71% similarUnverifiedQwen3.5-9B contains approximately 9 billion trainable parameters, and after 4-bit quantization, the model file size can be compressed to approximately 5–6 GB.71% similarUnverifiedFull fine-tuning of an SD XL model requires more than 80 GB of VRAM, whereas LoRA training achieving equivalent results can be completed on a consumer-grade GPU with 24 GB of VRAM.69% similarUnverifiedOn an A100 (80 GB VRAM) GPU, the throughput for training a 7B model is approximately 3,000–5,000 tokens per second.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/920061API
curl https://kongchang.com/api/v1/knowledge/claims/920061MCP
get_claim(id=920061)