Unverified50% confidenceFactExact time
Q4_K_M is a GGUF quantization format widely used in the llama.cpp community, in which weights are compressed to about 4 bits using the medium tier of the k-quant mixed quantization scheme, striking a compromise between size and precision.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedQ3 quantization refers to compressing model weights from FP16 format to an integer representation averaging approximately 3 bits.72% similarUnverifiedThe GGUF quantized version Q4_K_M of MiniCPM 2B has a model size of only 1.56G and can run smoothly on devices with 4G VRAM.72% similarUnverifiedGGUF 量化格式可将FP16或BF16精度模型压缩至4-bit乃至2-bit精度71% similarUnverifiedQ4 is the optimal choice for the vast majority of users, and 24GB VRAM paired with the Q4_K_M version is the sweet spot combination of quality and VRAM usage.70% similarUnverified在12GB显存的显卡上运行9B模型时，通常采用4-bit量化（如Q4_K_M格式）以降低显存占用70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1017042API
curl https://kongchang.com/api/v1/knowledge/claims/1017042MCP
get_claim(id=1017042)