Modality Gap

A phenomenon observed in multimodal retrieval, where embeddings from different modalities (such as text and images) tend to cluster in distinct cone-shaped regions within a high-dimensional shared vector space, even after contrastive learning alignment. This geometric separation causes cross-modal similarity scores to be systematically lower than within-modality scores.