How Does Near-Duplicate Image Selection Covertly Transmit Model Preferences?

Training data curation for vision-language models (VLMs) is often viewed as a neutral technical operation—deduplicating, picking the sharper version, keeping one image according to some heuristic rule. But a new study reveals that even choosing between nearly identical image copies can quietly transfer the curator's preferences to downstream models. This phenomenon, called "subliminal learning," was previously observed mainly in text data and has now been shown to exist in image selection as well.

What Is Subliminal Learning?

Subliminal learning refers to the phenomenon in which a model transmits its own tendencies to another model through data that does not explicitly express preferences. As models increasingly curate training data for other models, a key question emerges: could this transmission originate from the selection of near-duplicate images?

The researchers' experimental design is quite elegant. For each original photo, the curator selects one of 8 near-duplicate variants based on the likelihood between the description and the image, without using any target-specific prompt throughout. The student model is trained only on the images selected by the curator, and its log-likelihood ratio between the target and the alternative is then evaluated via an image-free readout.

• Near-duplicate images: Image copies that are nearly identical in content but differ subtly (e.g., slight cropping, compression artifacts, color shifts, rescaling, or re-encoding). Such copies abound in web-crawled datasets, and deduplication typically requires keeping one of them. The key to the study is that different copies differ slightly in how well they match the text description, so "which one to keep" itself constitutes an implicit selection signal.

• Image-free readout: A method for evaluating model preferences in which no image input is provided at the evaluation stage; the model is simply asked to score text descriptions, thereby comparing the log-likelihood of the target concept versus an alternative concept. This method is designed to exclude the direct influence of image content itself and to test whether the preference has been internalized into the model's language prior, and is therefore regarded as a sensitive probe for detecting subliminal learning.

Experimental Evidence: Directional Transfer of an Owl Preference

With "owl" as the target, student models trained on data selected by a curator that prefers owls showed a higher target-versus-alternative likelihood ratio in the image-free readout. According to the predefined decision rule, this directional shift appeared on LLaVA-1.5, Qwen2.5-VL, Qwen3-VL, and Gemma 4, with the curator and student initialized from the same base checkpoint.

More critically, when the target was rotated, the preference migrated accordingly to camels, oak trees, and a political figure, showing that this transmission is not an owl-specific coincidence. Across three models, students with the owl-target selection scored higher than all quantity-matched randomly reselected students; in LLaVA-1.5, higher than 55 of 60. Qwen3-VL reproduced the shift on a new source image pool and a newly trained curator.

The magnitude of the shift is affected by dataset size, learning rate, and training budget. With greedy selection among eight options, Qwen3-VL moved 10.4 percentage points toward the target, while other models moved 0.5 to 1.3 percentage points. A linear probe classified the selection arm with up to 59.7% accuracy (chance level 50%), indicating that the selection behavior leaves a detectable trace.

• Linear probe: An interpretability analysis tool that trains a simple linear classifier on top of a layer's activations in a model to determine whether that layer's representation linearly encodes specific information (e.g., "which variant was selected"). If the probe's accuracy is significantly above chance (50% for binary classification), the selection behavior has left a detectable trace in the representation space. The highest accuracy of 59.7% in this study was derived this way, indicating that selection preferences are not entirely unobservable.

• Base checkpoint: The same pretrained model weights from which both the curator model and the student model are initialized. Starting from the same point rules out confounding from differences in initial parameters, making the observed preference shift more likely attributable to how the training data was selected rather than to differences in model architecture or pretraining source.

Intergenerational Transmission and Implications for Deduplication

The first-generation student can also pass the preference to a second generation through descriptions that do not explicitly mention owls—verified in both LLaVA-1.5 and Qwen3-VL. This means preference transmission can accumulate across training generations without relying on any explicit label or prompt.

For everyday data pipelines, this finding challenges the intuition that "deduplication is a safe operation." Even if the source photos and human-written descriptions remain unchanged, merely choosing among near-duplicate copies can change the student model's relative readout of the target. Every choice a data curator makes, however mechanical it may seem, can become a vehicle for preference transmission.

Practical Recommendations

The study suggests three lines of defense worth attention: first, introduce randomization or diversity constraints into near-duplicate image selection to prevent a single heuristic rule from systematically favoring a particular target; second, incorporate statistical tests of selection behavior, such as linear probe detection, into data auditing; third, when training across generations, track implicit signals at the description level rather than only checking explicit content. These measures help reduce the risk of unintended preference transmission while maintaining data quality.

One-line Summary

Near-duplicate image selection itself can quietly transmit curator preferences to vision-language models.

Paragraph Summary

Research reveals that near-duplicate image curation in vision-language model training is not a neutral operation. In experiments, a curator model selected one of 8 near-duplicate variants of each original image based on image-text likelihood; the student model was trained only on that data, and its preferences were then evaluated via an image-free readout. Results show that with an owl as the target, the student model's target-versus-alternative likelihood ratio shifted directionally, holding on LLaVA-1.5, Qwen2.5-VL, Qwen3-VL, and Gemma 4; when the target was rotated, the preference migrated accordingly. Preferences can also be passed to a second-generation student through descriptions that do not explicitly mention the target. A linear probe's peak accuracy of 59.7% indicates that selection behavior leaves detectable traces. The study accordingly recommends introducing randomization into near-duplicate selection, adding statistical tests to data auditing, and tracking implicit signals at the cross-generational description level.