ConceptGradient Distortion
Gradient Distortion
A phenomenon in contrastive learning in which gradient weighting applied to highly similar negative examples harms downstream generalization performance. The model is pushed excessively away from certain samples, thereby破坏 the overall structure of the embedding space.
Timeline (last 90 days)
Oct 14
Gradient weighting on high-similarity negatives persistently harms downstream generalization, i.e., excessively pushing the model away from certain samples disrupts the overall structure of the embedding space, so retrieval performance drops rather than rises (termed 'gradient distortion').
Unverified50%