Unverified50% confidenceFactExact time
For a 750,000-token contract document, the original KV-cache is about 90GB, which after the HSQ-A compression pipeline drops to the order of about 10GB and can fit into the remaining VRAM of an A100-40GB.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
UnverifiedHSQ-A uses a lightweight scoring network to select about 15% of high-impact keys and stores the remaining values in FP8 format, thereby compressing the KV-cache from the 0.8 GB/k common in dense 70B models to 0.12 GB/k, a compression ratio of about 85%.75% similarVerifiedTaking LLaMA-2 70B as an example, when processing a 4096-token context with a batch size of 32, the KV Cache can consume over 80 GB of memory.72% similarVerifiedThe KV Cache of a 70B-parameter model with 128K context support can require hundreds of gigabytes of VRAM at full load, exceeding the 80GB VRAM capacity of a single H100.70% similarUnverifiedFor 1 million tokens, the KV Cache alone can consume hundreds of GB of VRAM at FP16 precision, far exceeding the 80 GB capacity limit of a single A100/H100.70% similarVerifiedLLaMA-2-70B's KV Cache for a single request at a context length of 128K can occupy more than 160 GB of VRAM.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1054135API
curl https://kongchang.com/api/v1/knowledge/claims/1054135MCP
get_claim(id=1054135)