Missing-Input Handling Mechanism
A design in multimodal models for handling missing modality inputs, enabling the model to still perform forward propagation and produce gradients when some modalities are unavailable. Common methods include missing markers and masked attention.
Timeline (last 90 days)
The paper observes that even when a certain modality is missing, the encoder can still adapt through a missing-input handling mechanism, so its updates should not simply be discarded.
The missing-input handling mechanism refers to a design whereby a multimodal model can still perform forward propagation and produce gradients when the input of a certain modality is unavailable, with common approaches including replacing that modality's embedding with a learnable missing token, zero-filling followed by masked attention to mask invalid positions, or introducing a modality-presence embedding.
All Facts (2)
The paper observes that even when a certain modality is missing, the encoder can still adapt through a missing-input handling mechanism, so its updates should not simply be discarded.
50%UnverifiedThe missing-input handling mechanism refers to a design whereby a multimodal model can still perform forward propagation and produce gradients when the input of a certain modality is unavailable, with common approaches including replacing that modality's embedding with a learnable missing token, zero-filling followed by masked attention to mask invalid positions, or introducing a modality-presence embedding.
50%