Unverified50% confidenceSolutionExact time
The correct practice is to complete all hyperparameter selection on an independent validation set, run the final evaluation on the test set only once, and report the results regardless of outcome.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
Unverified该基准测试使用确定性检查器（deterministic checker）评分，答案要么对要么错，不使用大模型当裁判78% similarUnverified厂商自行发布的基准测试往往存在测试集选择偏向、评判标准宽松、允许多次重试等方法论问题，需要独立第三方验证75% similarUnverifiedReliable evaluation protocols typically require pre-registration of the evaluation plan, explicitly stating whether only a single result or a multi-run average will be taken.75% similarUnverifiedThe evaluation used only one prompt, ran each model only once, used greedy decoding, scored answers based on normalized exact match, and did not score evidence_ids.75% similarUnverified在测试领域确定性意味着相同输入必须产生相同输出，同一测试在相同应用状态下运行必须得出一致结论74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1005190API
curl https://kongchang.com/api/v1/knowledge/claims/1005190MCP
get_claim(id=1005190)