Unverified50% confidenceFactExact time
An induction head is usually implemented by two attention heads working together: the previous-token head marks each token's predecessor, and the induction head uses that marker to retrieve the successor.
1
Sources
50%
Confidence
Long-term
Relevance
11/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedHead-level score modulation means assigning independent modulation parameters or functions to each attention head in multi-head attention, applying different transformations to the attention scores rather than having all heads share a single structural injection scheme.68% similarUnverifiedInduction heads are a special class of attention heads first systematically described by Anthropic's mechanistic interpretability research.65% similarUnverifiedCompared with adding a uniform edge bias to the scores, head-level score modulation allows different heads to attend to different types of edge relationships or subgraph patterns, enhancing the model's ability to express heterogeneous structural information and being naturally compatible with the block-wise computation flow of standard attention kernels.63% similarUnverifiedThe same attention head can perform entirely different sub-functions across different tasks.61% similarUnverifiedModern face recognition systems are typically divided into two separate stages: face detection and face recognition/matching.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1164449API
curl https://kongchang.com/api/v1/knowledge/claims/1164449MCP
get_claim(id=1164449)