#anisotropy
2 related articles
·6 min
DINOv2 vs SigLIP: Why the Huge Gap in k-NN Classification? A Guide to Avoiding Vision Encoder Selection Pitfalls
Experiments show DINOv2 Giant scores just 41% on k-NN classification, while SigLIP2 reaches 92%. This article dives into the embedding-space differences between contrastive and self-supervised learning to guide vision encoder selection.
Read more →
·7 min
Dispersion Loss: A Key Technique for Solving Embedding Condensation in Small Language Models
Embedding condensation is a hidden bottleneck in small language model training. Dispersion Loss combats this by enforcing representation spread during training at zero inference cost.
Read more →