Unverified50% confidenceFactExact time
Decide Live performs average pooling over the attention mask and then L2 normalization to produce a single 768-dimensional unit vector.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.68% similarUnverifiedLive Portrait's model weights are approximately 2–3 GB, and 6 GB of VRAM is sufficient to fully load them while retaining enough space to process intermediate feature maps.65% similarVerifiedFor a typical 7B model with 32 layers and 32 attention heads, storing the KV Cache for each token requires approximately 0.5 MB of VRAM, so generating 1,000 tokens requires an additional ~500 MB of VRAM62% similarUnverifiedWhen sequence length n grows from 2K to 128K, the attention computation volume increases approximately 4,000-fold.60% similarUnverifiedEngram dedicates one quarter of the entire checkpoint's capacity to n-gram memory, reaching a scale of 197B parameters.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1002776API
curl https://kongchang.com/api/v1/knowledge/claims/1002776MCP
get_claim(id=1002776)