Unverified50% confidenceFactExact time
TopK SAE keeps the largest several pre-activation values on each token, and the margin of a feature is defined as the distance between its pre-activation value and the truncation threshold.
1
Sources
50%
Confidence
Long-term
Relevance
11/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTop-K SAE retains only the K features with the highest activation values during encoding and forces all others to zero, thereby compelling the model to learn sparse representations.72% similarUnverifiedThe K value selection for Top-K SAE is typically between 32 and 256, depending on the model scale and the granularity requirements of the analysis.63% similarUnverifiedtop_k directly truncates to the k most probable candidate tokens, and the smaller k is, the more concentrated the output.61% similarVerifiedTop-P (nucleus sampling) samples from the smallest set of tokens whose cumulative probability exceeds the threshold P, while Top-K selects only from the K candidates with the highest probabilities.60% similarUnverifiedThe value of K in Top-K should be determined through offline evaluation by measuring Recall@K at different K values on an annotated test set and finding the inflection point where the recall curve begins to flatten.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129396API
curl https://kongchang.com/api/v1/knowledge/claims/1129396MCP
get_claim(id=1129396)