Unverified50% confidenceFactExact time
vLLM's FP8 KV cache adds 0.0004 to 0.0006 of KLD at every tested checkpoint on Qwen3.8 27B, a very small value that nonetheless appears consistently on almost every prompt.
1
Sources
50%
Confidence
Long-term
Relevance
11/7/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn tests on Qwen3.8 27B, even with activations kept in bf16, vLLM's NVFP4 checkpoint only matches UD-Q5_K_XL in KLD, while the latter is 8.7 GiB smaller in size.75% similarUnverifiedIn tests on Qwen3.8 27B, the absolute KLD increase of the q4_0 KV cache harms more precise weights more: Q3-level weights increase by only 6-10%, UD-Q4_K_XL increases by about 39%, and Q8_0 increases by about 390%.74% similarUnverifiedIn the same test, using the Marlin kernel to keep quantized weights while keeping activations in bf16 lowered Qwen FP8's KLD from 0.0039 to 0.0028, a reduction of about 30%.72% similarUnverifiedThe dense Qwen3.8-27B model's 3-bit version actually scored 0.05 higher (within noise range), but the number of tokens generated per task increased by 40% (confidence interval 1.25–1.59x).70% similarUnverifiedThe Q3_XXS model paired with an F16 KV cache scores approximately 80% on the needle test, while Q3_K_XL paired with a kvarn3 KV cache scores close to 100%.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1146817API
curl https://kongchang.com/api/v1/knowledge/claims/1146817MCP
get_claim(id=1146817)