Q-Former
Q-Former is a bridging module proposed by Salesforce in BLIP-2. It uses learnable query vectors to extract information from the output of a vision encoder through cross-attention, compressing an arbitrary number of image features into a fixed number of tokens.
Timeline (last 90 days)
Learned resamplers (such as Q-Former) introduce additional parameters and attention computation overhead when compressing the number of tokens.
Q-Former is a bridging module proposed by Salesforce in BLIP-2 that uses learnable query vectors to extract information from the output of a visual encoder through cross-attention, compressing it into a fixed number of tokens.
Alignment between visual embedding vectors and text embedding vectors is typically accomplished through a projection layer, with common implementations including a linear layer, MLP, Q-Former (as used in BLIP-2), or Perceiver Resampler (as used in Flamingo).
Q-Former extracts a fixed number of visual features from a frozen visual encoder through a set of learnable query vectors, achieving information bottleneck-style compression.
Common implementations of the cross-modal alignment module include a linear projection layer (used by LLaVA), Q-Former (used by BLIP-2), and cross-attention mechanisms.
BLIP-2 uses a Q-Former structure to extract a fixed number of visual features through learnable query vectors.
All Facts (6)
BLIP-2 uses a Q-Former structure to extract a fixed number of visual features through learnable query vectors.
60%UnverifiedLearned resamplers (such as Q-Former) introduce additional parameters and attention computation overhead when compressing the number of tokens.
50%UnverifiedQ-Former is a bridging module proposed by Salesforce in BLIP-2 that uses learnable query vectors to extract information from the output of a visual encoder through cross-attention, compressing it into a fixed number of tokens.
50%UnverifiedAlignment between visual embedding vectors and text embedding vectors is typically accomplished through a projection layer, with common implementations including a linear layer, MLP, Q-Former (as used in BLIP-2), or Perceiver Resampler (as used in Flamingo).
50%UnverifiedQ-Former extracts a fixed number of visual features from a frozen visual encoder through a set of learnable query vectors, achieving information bottleneck-style compression.
50%UnverifiedCommon implementations of the cross-modal alignment module include a linear projection layer (used by LLaVA), Q-Former (used by BLIP-2), and cross-attention mechanisms.
50%