SigLIP

The Gemma series of models uses the SigLIP vision encoder to process image inputs.

SigLIP is a vision-language contrastive learning model proposed by Google Research that uses a sigmoid loss function to replace CLIP's softmax contrastive loss, eliminating the dependency on global negative samples.

Improved CLIP variants such as SigLIP and OpenCLIP further advance accuracy and multilingual support.

Mainstream VLMs typically use a pretrained Vision Transformer (ViT) as the visual encoder, such as CLIP ViT and SigLIP.

SigLIP replaces the softmax loss with a sigmoid loss, performs better at larger batch sizes, and has been widely adopted by Google's Gemini and other multimodal models.

SigLIP stands for Sigmoid Loss for Language-Image Pre-training.

SigLIP retains the core features of CLIP, including the image encoder, text encoder, embedding representations, and similarity matrix; the true difference between the two lies in the design of the loss function.

SigLIP abandons global normalization and treats each image-text pair as an independent binary prediction, using the sigmoid function to estimate whether the image and text match.

The visual capabilities of multimodal models are typically implemented through a visual encoder, with common approaches including pre-trained visual models such as ViT or SigLIP.

CLIP uses softmax cross-entropy loss with a computational complexity of O(n²), while SigLIP's sigmoid loss allows each image-text pair to be computed independently, eliminating the hard dependency on large-batch training.

SigLIP is an improved version of CLIP that replaces softmax with a sigmoid loss function, making it more efficient when contrasting sample pairs within a batch.

Moondream uses SigLIP as its visual encoder.

OpenCLIP 基于 LAION 数据集训练，SigLIP 是 Google 的改进版本，都允许商用且可进行领域微调

PaliGemma uses SigLIP as the vision encoder and is a combined architecture of a vision encoder and a language model.