Unverified50% confidenceDecision RuleExact time
The K value selection for Top-K SAE is typically between 32 and 256, depending on the model scale and the granularity requirements of the analysis.
1
Sources
50%
Confidence
Long-term
Relevance
8/10/2026
First Seen
Sources
VHectorLab 3D：开源3D可视化工具探索LLM潜空间结构
Related Claims
UnverifiedTop-K SAE retains only the K features with the highest activation values during encoding and forces all others to zero, thereby compelling the model to learn sparse representations.64% similarUnverifiedK2.6 uses only approximately 3% of its total parameters per inference, making its inference cost comparable to that of a 32B dense model.62% similarVerifiedTop-P (nucleus sampling) samples from the smallest set of tokens whose cumulative probability exceeds the threshold P, while Top-K selects only from the K candidates with the highest probabilities.61% similarUnverifiedThe developer ultimately selected the variant with a KL divergence of 0.028 as the optimal solution.61% similarUnverified2.4万亿参数模型在BF16精度下考虑KV Cache和激活值后实际约需1.4-1.5TB显存60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/728238API
curl https://kongchang.com/api/v1/knowledge/claims/728238MCP
get_claim(id=728238)