Unverified50% confidenceOpinionExact time
Data provenance and cleaning are the first line of defense against pretraining poisoning, but pretraining data is enormous in scale (often reaching the TB level), full manual review is unrealistic, and automated cleaning may miss carefully disguised malicious samples.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedPretraining backdoors differ from common fine-tuning poisoning: the attack occurs in the early stage when the model learns general language capabilities, and once poisoned data is mixed into a massive corpus it is extremely difficult to audit item by item, and subsequent safety alignment and fine-tuning often cannot thoroughly remove it.77% similarUnverifiedData engineering also includes sample deduplication (to avoid training set leakage), difficulty stratification (to prevent simple samples from dominating the gradient), and quality filtering (to remove low-quality or harmful content).73% similarVerifiedBenchmark data contamination refers to the phenomenon where, after a test set is made public, its questions may appear in the pre-training corpora of subsequent models, causing scores to reflect memorization rather than genuine reasoning ability.70% similarUnverifiedLEAPS adopts a data-contamination-resistant design, with datasets built entirely anew to avoid inflated scores due to models' pretraining memory.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1127998API
curl https://kongchang.com/api/v1/knowledge/claims/1127998MCP
get_claim(id=1127998)