Vision Transformer
A visual model that applies the Transformer architecture to image processing. An input image is divided into fixed-size patches, which are then treated as a sequence and fed into a Transformer encoder. Vision Transformer serves as the core backbone network for Diffusion Transformer (DiT) and related architectures.
Core Facts
The Vision Transformer (ViT) was proposed by Google in 2020, dividing images into fixed-size patches as sequence inputs to a Transformer encoder.
Vision Transformer (ViT) was proposed by the Google Brain team in 2020, introducing the Transformer architecture into the field of computer vision.
ViT (Vision Transformer) splits an image into a sequence of fixed-size 16×16 patches before feeding them into a standard Transformer.
Timeline (last 90 days)
The genealogy of computer vision models is more miscellaneous than that of language models, including LeNet, AlexNet, VGG, ResNet, DenseNet, EfficientNet, Vision Transformer, Swin, ConvNeXt, and others, with each architecture solving a specific defect exposed by the previous architecture.
The [CLS] token was introduced by BERT; in vision Transformers it is prepended to the sequence of patch tokens and aggregates global image semantic information through self-attention, but the aggregation process discards a large amount of spatial local information.
When an image contains rendered text, the model invokes an independent perception-to-text routing pathway rather than the metonymic anchor channel.
Experiments reveal that the model's early layers are dominated by perceptual primitives, primarily processing low-level visual features such as color, shape, and texture.
The arXiv paper "Metonymic Circuits for Abstract Concept Grounding in Vision Transformers" proposes that vision Transformers understand abstract concepts through a metonymic grounding mechanism.
These depth estimation models typically use Vision Transformer backbone networks and are pre-trained on millions to billions of images
The concept of register tokens was originally proposed by Darcet et al. in Vision Transformer research to address the problem of abnormally high activation values appearing in certain patch tokens in ViTs.
The MAE proposed by He et al. in 2021 splits images into patches, randomly masks up to 75% of the patches, encodes the visible patches with a Vision Transformer, and reconstructs the masked portions.
CAPI is a self-supervised method focused on patch-level representation learning, constructing pre-training objectives at the patch token level of Vision Transformers.
GPT-6 Astra uses a Vision Transformer (ViT) architecture for visual processing.
16 more timeline events
All Facts (20)
The Vision Transformer (ViT) was proposed by Google in 2020, dividing images into fixed-size patches as sequence inputs to a Transformer encoder.
90%VerifiedVision Transformer (ViT) was proposed by the Google Brain team in 2020, introducing the Transformer architecture into the field of computer vision.
90%VerifiedViT (Vision Transformer) splits an image into a sequence of fixed-size 16×16 patches before feeding them into a standard Transformer.
80%VerifiedThe typical architecture of a VLM (Vision-Language Model) consists of three core components: a visual encoder (such as ViT), a language model (typically a large language model), and a projection layer or adapter module connecting the two.
70%VerifiedVision Transformer divides images into 16×16 patches as the basic processing unit.
70%VerifiedImage recognition in agricultural scenarios is primarily based on convolutional neural networks (CNN) in deep learning and their variants, such as model architectures including ResNet, EfficientNet, and Vision Transformer.
70%VerifiedVision Transformer (ViT) emerged in 2020, marking the entry of Transformers into the field of computer vision.
65%VerifiedVision Transformer (ViT) introduced the self-attention mechanism from the NLP domain into image understanding, processing images by splitting them into patches and treating them as sequences.
65%UnverifiedTransformer架构被引入视觉生成（如Vision Transformer、DiT架构），使模型能更好理解图像不同区域间的全局关系，适用于多模态场景中文本token和图像token的统一处理
88%UnverifiedIn the Vision Transformer architecture, a standard 224×224 image split with a patch size of 14×14 produces 256 visual tokens.
70%UnverifiedGPT-6 Astra uses a Vision Transformer (ViT) architecture for visual processing.
60%UnverifiedSAM's architecture consists of three components: a heavyweight image encoder based on Vision Transformer, a flexible prompt encoder, and a lightweight mask decoder.
60%UnverifiedCLIP contains two parallel encoders: the text encoder uses a Transformer-based architecture (a 12-layer Transformer similar to GPT-2), and the visual encoder has two variants—one based on ResNet and one based on ViT.
60%UnverifiedThe visual understanding of modern multimodal models is typically based on the ViT architecture, where images are divided into fixed-size patches (usually 16×16 or 32×32 pixels) and linearly projected into vector embeddings.
60%UnverifiedLayoutLM and Donut are representative specialized document understanding models.
60%UnverifiedThe genealogy of computer vision models is more miscellaneous than that of language models, including LeNet, AlexNet, VGG, ResNet, DenseNet, EfficientNet, Vision Transformer, Swin, ConvNeXt, and others, with each architecture solving a specific defect exposed by the previous architecture.
50%UnverifiedThe [CLS] token was introduced by BERT; in vision Transformers it is prepended to the sequence of patch tokens and aggregates global image semantic information through self-attention, but the aggregation process discards a large amount of spatial local information.
50%UnverifiedWhen an image contains rendered text, the model invokes an independent perception-to-text routing pathway rather than the metonymic anchor channel.
50%UnverifiedExperiments reveal that the model's early layers are dominated by perceptual primitives, primarily processing low-level visual features such as color, shape, and texture.
50%UnverifiedThe arXiv paper "Metonymic Circuits for Abstract Concept Grounding in Vision Transformers" proposes that vision Transformers understand abstract concepts through a metonymic grounding mechanism.
50%