Unverified50% confidenceBenchmarkExact time
Actual testing of two quantization models: IQ2_XXS achieves 48 tokens/s for short Q&A and 63 tokens/s for long text; IQ3_XXS achieves 30 tokens/s for short Q&A and 39 tokens/s for long text, with VRAM usage of approximately 16.44 GB, activating 10,138 experts—IQ2 is about 60% faster than IQ3.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/5/2026
First Seen
Valid until: 1/3/2027
Sources
Related Entities
Related Claims
UnverifiedIQ2_XXS以约2.06位的平均比特率编码权重，IQ3_S约3.44位，Q4_K_M约4.83位71% similarUnverifiedThe fine-tuned 406M model achieved 36.33 ROUGE-1 on the test set, while the authors' own 1.2B system achieved 35.41 ROUGE-1, and the two cannot be statistically distinguished.67% similarUnverifiedIn the MoE architecture, the Mixtral 8x7B model has a total of 46.7B parameters, but only approximately 12.9B parameters (2 experts) are activated per token, making inference costs close to those of a 13B dense model while achieving performance comparable to the 70B level.66% similarUnverifiedPhi-2 has 2.7B parameters and on multiple benchmark tests can match models several times its size.66% similarUnverifiedIn Strata's speed table, under a 128k context, Q2_0 (2-bit) achieves 65.1 tokens/s and IQ2_XS achieves 52 tokens/s; under a 4k short context, IQ3_XXS achieves 65.6 tokens/s.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/978738API
curl https://kongchang.com/api/v1/knowledge/claims/978738MCP
get_claim(id=978738)