DINOv2-base
A self-supervised vision Transformer model proposed by Meta, based on the ViT-B/14 architecture with approximately 86 million parameters. It is pretrained on large-scale unlabeled images using a self-distillation objective and produces high-quality general-purpose visual features without text supervision.
Timeline (last 90 days)
On the image side, a frozen DINOv2-base was used and embedded only once; on the text side, a uniform 4-layer, width-256 encoder was used, with about 11 million parameters; the text model never had access to images throughout.
The experiments cover only one dataset (COCO) and one image model (DINOv2-base), and the best learning rate for next-word prediction falls exactly at the upper bound of the search range, so better unexplored configurations may exist.
DINOv2-base is a self-supervised vision Transformer model proposed by Meta, based on the ViT-B/14 architecture with about 86 million parameters, pre-trained on large-scale unlabeled images via a self-distillation objective, producing general-purpose visual features without text supervision.
All Facts (3)
On the image side, a frozen DINOv2-base was used and embedded only once; on the text side, a uniform 4-layer, width-256 encoder was used, with about 11 million parameters; the text model never had access to images throughout.
50%UnverifiedThe experiments cover only one dataset (COCO) and one image model (DINOv2-base), and the best learning rate for next-word prediction falls exactly at the upper bound of the search range, so better unexplored configurations may exist.
50%UnverifiedDINOv2-base is a self-supervised vision Transformer model proposed by Meta, based on the ViT-B/14 architecture with about 86 million parameters, pre-trained on large-scale unlabeled images via a self-distillation objective, producing general-purpose visual features without text supervision.
50%