Unverified50% confidenceSolutionExact time
vLLM's ModelRunner uses the int8 type to allocate untyped byte arrays on a per-byte basis when allocating KV Cache VRAM.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/10/2026
First Seen
Valid until: 12/9/2026
Sources
Related Entities
Related Claims
UnverifiedvLLM uses PagedAttention technology to manage KV Cache, and its memory allocation and request scheduling introduce runtime non-determinism.63% similarUnverifiedvLLM adopts a scheduling-execution separation architecture, where the scheduling side is responsible for request management, sequence scheduling, and logical resource allocation, while the execution side is responsible for actual model inference and physical VRAM management.63% similarUnverifiedThere is a bug in vLLM when prefix caching and DFlash2 are both enabled, which causes the last cache unit (a mixed-layout block of approximately 16 tokens) to be directly discarded and recomputed.63% similarUnverifiedOutlines operates at the token probability level and is compatible with multiple open-source model backends such as vLLM, llama.cpp, and Transformers, without depending on a specific vendor's API.61% similarUnverifiedThe new models Sol and Terra are forcibly routed to V2 via the subagent version field in the models cache JSON file, regardless of user configuration.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/887055API
curl https://kongchang.com/api/v1/knowledge/claims/887055MCP
get_claim(id=887055)