Unverified50% confidenceConceptExact time
Benchmark Selection Bias refers to the tendency of vendors, when choosing evaluation items themselves, to preferentially include task types for which their own models have been specifically optimized, thereby creating a favorable tilt in the overall score.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThis identification bias is statistically known as Selection Bias, where the sample is determined by observable characteristics rather than being randomly drawn from the overall population.73% similarUnverified'Measurement Bias' in questionnaire design refers to the way question wording, option order, and scale extremity systematically influence responses, skewing the distribution toward the direction preset by the designer.71% similarUnverifiedSelf-selection bias means survey data recruited through social platforms or voluntary questionnaires is suitable for discovering hypotheses, refining dimensions, and identifying extreme cases, but not for estimating demand size or adoption rates.71% similarUnverifiedThere exists an evaluation-application gap between benchmark performance and actual production environment performance, and some vendors tend to over-optimize for evaluation sets (benchmark gaming / benchmark contamination).70% similarUnverifiedA common limitation shared by mainstream bias benchmarks is that they measure a model's explicit choices in specific controlled contexts, rather than implicit biases that emerge in open-ended generation, long-text reasoning, or complex role-playing scenarios.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1036643API
curl https://kongchang.com/api/v1/knowledge/claims/1036643MCP
get_claim(id=1036643)