LM head
The LM head is the final decoding module in a language model that maps internal representations to a probability distribution over tokens. In audio language models it is responsible for converting encoded features into text output.
Timeline (last 90 days)
Through linear probing analysis, VocalGrad research found two failure modes in LALMs: representations exist but the LM head does not exploit them, and information gradually becomes non-linearly decodable as it propagates across layers.
The LM head is the final decoding module in a large audio language model that maps internal representations to a probability distribution over text tokens.
All Facts (2)
Through linear probing analysis, VocalGrad research found two failure modes in LALMs: representations exist but the LM head does not exploit them, and information gradually becomes non-linearly decodable as it propagates across layers.
50%UnverifiedThe LM head is the final decoding module in a large audio language model that maps internal representations to a probability distribution over text tokens.
50%