Unverified50% confidenceBenchmarkExact time
After Q4_K_M quantization, the pruned 3.81B model is 2.75 GB in size, achieves an inference speed of 79 tokens/s on a Mac, and uses about 3.8 GB of memory.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
Related Entities
Related Claims
UnverifiedTesting with a 4-bit quantized Qwen2.5 7B model on an M2 Mac mini with 16 GB unified memory, throughput dropped from 16.8 tokens/s without batching to 8.7 tokens/s with batching.80% similarVerified4-bit quantization (such as Q4_K_M) can typically compress the memory footprint of a 3B model from approximately 6GB to approximately 2GB, with a precision loss of less than 2–3%.77% similarVerifiedQ4_K_M uses mixed precision, storing some critical weights at 6-bit and the remainder at 4-bit, with an overall average of approximately 4.5-bit.76% similarUnverifiedWhen the author ran Qwen-Image-2.1 at 1024×1024 resolution on the same Mac, the process footprint never exceeded 18.5 GB, but swap grew by 3.5 GB during the denoising stage and by 9.6 GB during the VAE decoding stage, and finally the memory guard actively terminated the run before writing out the image.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1136033API
curl https://kongchang.com/api/v1/knowledge/claims/1136033MCP
get_claim(id=1136033)