HLR-LLM
An audio-visual large language model with spatial awareness. On top of a monaural semantic audio pathway, it adds a binaural spatial audio branch, combining curriculum learning and chain-of-thought supervision to achieve semantic understanding of sound and spatial localization reasoning.
Timeline (last 90 days)
The research team proposed HLR-LLM, which, while retaining the original monaural semantic audio pathway, adds an additional binaural spatial audio branch.
HLR-LLM is trained with a curriculum learning strategy and introduces chain-of-thought supervision in the final stage, specifically targeting multi-step spatial reasoning tasks.
Experimental results show that HLR-LLM achieves significant improvements in spatial audio-visual scene understanding without harming semantic audio-visual perception capabilities.
All Facts (3)
HLR-LLM is trained with a curriculum learning strategy and introduces chain-of-thought supervision in the final stage, specifically targeting multi-step spatial reasoning tasks.
50%UnverifiedExperimental results show that HLR-LLM achieves significant improvements in spatial audio-visual scene understanding without harming semantic audio-visual perception capabilities.
50%UnverifiedThe research team proposed HLR-LLM, which, while retaining the original monaural semantic audio pathway, adds an additional binaural spatial audio branch.
50%