Unverified50% confidenceBenchmarkExact time
An EMNLP study found that 4-bit models suffer an average retrieval loss of 23% at 128,000 tokens, with 70B LLaMA losing one-third of its score while QWEN of the same scale loses almost none.
1
Sources
50%
Confidence
Long-term
Relevance
9/16/2026
First Seen
Sources
Related Entities
Related Claims
Verified4-bit quantization reduces model size to 1/8 of the original at the cost of approximately 1–3% accuracy loss.73% similarVerifiedFor models with more than 30B parameters, the performance loss caused by 4-bit quantization typically results in a 1–3% drop on benchmarks, but the impact is more significant for smaller models.71% similarVerifiedINT8 quantization loses approximately 0.5% accuracy while halving VRAM usage; INT4 quantization compresses VRAM to 1/4 of the original with an accuracy loss of approximately 1–3%.70% similarUnverifiedOLLAMA_KV_CACHE_TYPE=q8_0 occupies approximately only half the memory of the f16 default, with minimal quality loss68% similarUnverifiedDuring epochs 29 to 50, the variance of the sign-pattern loss is only ±0.40%, whereas the magnitude-based variant reaches as high as ±2.25%66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/925955API
curl https://kongchang.com/api/v1/knowledge/claims/925955MCP
get_claim(id=925955)