Unverified50% confidenceSolutionExact time
使用IQ4_XS配合-fa 1 -ctk q8_0 -ctv q8_0对KV缓存进行q8_0量化后，可加载完整32k上下文，常驻内存约16.6GB
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/20/2026
First Seen
Valid until: 11/18/2026
Sources
Related Claims
Unverified以Q4_0格式为例，每32个FP32权重被打包为一个量化块，包含一个缩放因子和32个4bit整数，平均每参数约占4.5bit，内存占用缩减至原来的约1/878% similarUnverified以约160GB的模型（如Qwen3.5-397B的UD-Q3_K_XL量化版本）为例，运行透镜需额外约20GB内存77% similarUnverifiedQwen-7B经过4-bit量化后，一张搭载24GB显存的消费级显卡即可流畅运行75% similarUnverifiedQ4_K_M量化文件大小17.77GB，pp2048速度96.8 tok/s，tg128速度11.4 tok/s75% similarUnverifiedLlama 3 70B量化版（Q4精度）仍需约35GB内存75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/778779API
curl https://kongchang.com/api/v1/knowledge/claims/778779MCP
get_claim(id=778779)