Unverified50% confidenceOpinionExact time
SWE-bench carries a risk of data contamination because the model's training data may already include the relevant issues and PRs.
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedData contamination refers to a situation where a model has already seen data from the test set during training, causing benchmark scores to be inflated.71% similarVerifiedData poisoning attacks involve mixing malicious samples into training datasets, causing the model to exhibit incorrect behavior toward specific inputs after deployment.70% similarVerifiedData leakage refers to test set information being indirectly seen by the model during training, causing evaluation metrics to be inflated; it is one of the most common and most concealed methodological flaws in machine learning research, and is especially dangerous in the medical prediction domain.68% similarUnverifiedThe greatest risk of using synthetic data lies in the potential lack of real-world relevance, seasonality, noise, and outliers, which can distort insights.65% similarVerifiedStandardizing the entire dataset before splitting it leads to data leakage, exposing test set information to the training set.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/976890API
curl https://kongchang.com/api/v1/knowledge/claims/976890MCP
get_claim(id=976890)