Induction Head
A type of specialized attention head in Transformers responsible for detecting previously occurring patterns and predicting their subsequent tokens. It is an important basis for in-context learning, but may also lead to repetition loops.
Timeline (last 90 days)
By scanning attention heads, the developer found that an induction head drives the loop, with an induction score of 0.425, while the ordinary copying score is only 0.012.
An induction head is usually implemented by two attention heads working together: the previous-token head marks each token's predecessor, and the induction head uses that marker to retrieve the successor.
Induction heads are a special class of attention heads first systematically described by Anthropic's mechanistic interpretability research.
That induction head is also responsible for copying names into reading-comprehension answers, so simply ablating it would impair reading ability.
After directly ablating that induction head, the loop rate fell from 29.4% to 25.4%.
In experiment 3, the researchers removed the input and output layers and kept only four Transformer blocks, and the LLM correctly determined all of the languages the model was trained on (Python, English, and romaji Japanese), and was able to locate the attention head responsible for copying (the induction head).
Mechanistic interpretability methods have been widely used in recent years to localize specific functional modules such as induction heads and fact retrieval circuits.
Induction heads are an important clue for understanding the capability of in-context learning.
Upon seeing [A][B]...[A], induction heads tend to predict that the next token is [B].
The research identifies induction heads, which can recognize patterns that have appeared previously in a sequence and predict their repetition.
1 more timeline events
All Facts (11)
By scanning attention heads, the developer found that an induction head drives the loop, with an induction score of 0.425, while the ordinary copying score is only 0.012.
50%UnverifiedAn induction head is usually implemented by two attention heads working together: the previous-token head marks each token's predecessor, and the induction head uses that marker to retrieve the successor.
50%UnverifiedInduction heads are a special class of attention heads first systematically described by Anthropic's mechanistic interpretability research.
50%UnverifiedThat induction head is also responsible for copying names into reading-comprehension answers, so simply ablating it would impair reading ability.
50%UnverifiedAfter directly ablating that induction head, the loop rate fell from 29.4% to 25.4%.
50%UnverifiedIn experiment 3, the researchers removed the input and output layers and kept only four Transformer blocks, and the LLM correctly determined all of the languages the model was trained on (Python, English, and romaji Japanese), and was able to locate the attention head responsible for copying (the induction head).
50%UnverifiedMechanistic interpretability methods have been widely used in recent years to localize specific functional modules such as induction heads and fact retrieval circuits.
50%UnverifiedInduction heads are an important clue for understanding the capability of in-context learning.
50%UnverifiedUpon seeing [A][B]...[A], induction heads tend to predict that the next token is [B].
50%UnverifiedThe research identifies induction heads, which can recognize patterns that have appeared previously in a sequence and predict their repetition.
50%UnverifiedOpenAI的研究团队通过可解释性方法发现了模型内部的归纳头(induction heads)等关键机制
50%