Think Before You Link
An arXiv paper on multimodal entity linking that proposes evaluating rare entities using knowledge-graph structural metrics and introduces a training-free framework combining reasoning and retrieval to address the challenge of rare-entity linking.
Timeline (last 90 days)
The arXiv paper "Think Before You Link" reveals that multimodal entity linking systems fail significantly when confronted with rare entities, even for the most advanced models.
On the rare entity slices newly partitioned by structural metrics, the accuracy of current state-of-the-art systems drops by 15.4% to 39.9%.
The researchers propose a simple, training-free framework that enables reasoning-capable vision-language models to perform iterative search, dynamic reasoning, and evidence accumulation.
Ablation experiments show that relying solely on reasoning yields no significant improvement in accuracy for rare entities.
Ablation experiments show that combining reasoning with retrieval yields the best performance, with the two forming a complementary relationship.
The framework achieves an overall accuracy improvement of 6.9% over the current state of the art on the MERLIN benchmark.
The framework achieves an accuracy improvement of up to 23.3% on rare entity slices.
All Facts (7)
The arXiv paper "Think Before You Link" reveals that multimodal entity linking systems fail significantly when confronted with rare entities, even for the most advanced models.
50%UnverifiedOn the rare entity slices newly partitioned by structural metrics, the accuracy of current state-of-the-art systems drops by 15.4% to 39.9%.
50%UnverifiedThe researchers propose a simple, training-free framework that enables reasoning-capable vision-language models to perform iterative search, dynamic reasoning, and evidence accumulation.
50%UnverifiedAblation experiments show that relying solely on reasoning yields no significant improvement in accuracy for rare entities.
50%UnverifiedAblation experiments show that combining reasoning with retrieval yields the best performance, with the two forming a complementary relationship.
50%UnverifiedThe framework achieves an overall accuracy improvement of 6.9% over the current state of the art on the MERLIN benchmark.
50%UnverifiedThe framework achieves an accuracy improvement of up to 23.3% on rare entity slices.
50%