Concept基准过拟合 / Goodhart's Law / 古德哈特定律
benchmark overfitting
A phenomenon in which an AI model is optimized through training on a specific benchmark and achieves high scores on the test set without a corresponding improvement in generalization ability. This behavior is considered a manifestation of Goodhart's Law within the context of machine learning evaluation.
Core Facts
Timeline (last 90 days)
Sep 11
Methods for overfitting benchmarks include mixing evaluation data into training corpora, indirect web-crawl contamination, instruction-style fine-tuning, and training models to recognize evaluation scenarios.
Verified65%