Perceiver Resampler
A learnable visual token resampling module that introduces a set of trainable queries and a cross-attention mechanism to compress an arbitrary number of visual tokens into a fixed number of output tokens. It requires additional training.
Timeline (last 90 days)
Alignment between visual embedding vectors and text embedding vectors is typically accomplished through a projection layer, with common implementations including a linear layer, MLP, Q-Former (as used in BLIP-2), or Perceiver Resampler (as used in Flamingo).
Flamingo was proposed by DeepMind in 2022 and uses a Perceiver Resampler and cross-attention layers to achieve deep fusion of vision and language.
All Facts (2)
Alignment between visual embedding vectors and text embedding vectors is typically accomplished through a projection layer, with common implementations including a linear layer, MLP, Q-Former (as used in BLIP-2), or Perceiver Resampler (as used in Flamingo).
50%UnverifiedFlamingo was proposed by DeepMind in 2022 and uses a Perceiver Resampler and cross-attention layers to achieve deep fusion of vision and language.
50%