Unverified50% confidenceFactExact time
Some ONNX exports labeled q4 are larger than the q4f16 versions because the embedding table retains 32-bit precision; for example, SmolLM2-360M's q4 is 390MB and q4f16 is 275MB, and the two produce identical outputs.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTaking the Q4_0 format as an example, every 32 FP32 weights are packed into one quantization block containing one scale factor and 32 4-bit integers, averaging approximately 4.5 bits per parameter, reducing memory usage to approximately 1/8 of the original.78% similarUnverified使用IQ4_XS配合-fa 1 -ctk q8_0 -ctv q8_0对KV缓存进行q8_0量化后，可加载完整32k上下文，常驻内存约16.6GB77% similarUnverifiedLlama 3 70B量化版（Q4精度）仍需约35GB内存76% similarVerifiedThe Qwen2.5:7B model is approximately 4 GB in size after Q4 quantization.76% similarUnverifiedEach of Qwen3-27B's 16 full-attention layers has 4 KV heads and a head dimension of 256, occupying 64 KB per token at 16-bit precision.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013586API
curl https://kongchang.com/api/v1/knowledge/claims/1013586MCP
get_claim(id=1013586)