Unverified50% confidenceFactExact time
With the 7B AWQ model at batch 32 the 3090 already lags behind the L40S (0.71x), whereas the 7B fp16 is not overtaken until batch 128 (0.72x), showing that the quantized model makes the compute bottleneck appear earlier.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
Unverified在27B混合架构、4B稠密模型、36B MoE三类模型上以8k上下文测试，Prefill阶段性能无变化，Decode阶段打补丁后快约1.4%64% similarUnverifiedFor the Qwen2.5-1.5B model, the int4 theoretical roofline ceiling for batch-1 decoding is 411.8 tok/s, while the actual self-developed kernel reaches only 25.5 tok/s, a gap of about 16x.63% similarUnverified7–8B is the minimum threshold for lx tools; llama3.1:8b and qwen2.5:7b perform well, and 14B-class models can basically achieve cloud API-level results.62% similarUnverifiedMistral 7B证明精心训练的7B模型可以匹敌早期的13B甚至30B模型62% similarUnverifiedUsing a 4B model to rerank the candidate shortlist filtered by BM25 outperformed BM25 alone (+0.047, confidence interval [0.020, 0.073]), even though the 4B reranker itself did not pass qualification.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1061231API
curl https://kongchang.com/api/v1/knowledge/claims/1061231MCP
get_claim(id=1061231)