Unverified50% confidenceFactExact time
The TopK sparse autoencoder retains only the K neurons with the highest activation values during each forward pass, setting all others to zero.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFor each generator, the key neurons consistently selected account for less than 1% of the total, and they are highly consistent across different cross-validation folds and random seeds.72% similarUnverifiedMean ablation of the key neurons leaves accuracy largely unchanged, revealing the redundant distributed nature of the detection signal.63% similarUnverifiedIn recurrent neural networks, the critical state corresponds to the maximum singular value (spectral radius) of the Jacobian matrix approaching 1.63% similarUnverifiedThe choice of K value in Top-K routing affects the trade-off between computational efficiency and expressive capacity — a smaller K leads to faster inference but may sacrifice expressiveness, and in practice K is typically set to 1 or 2.62% similarUnverifiedThe top 20 decoder layers do not read the prompt; their global key-values are projected from the encoder's final hidden states via per-layer independent weight matrices, reducing prefill complexity to nearly half (n×L/2).62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990358API
curl https://kongchang.com/api/v1/knowledge/claims/990358MCP
get_claim(id=990358)