Unverified50% confidenceOpinionExact time
The pass@k metric does not penalize redundant or incorrect reasoning paths and is sensitive to the value of k, so in practice it is often combined with multidimensional metrics such as majority-vote accuracy, reasoning-chain step correctness, and human evaluation.
1
Sources
50%
Confidence
Long-term
Relevance
10/24/2026
First Seen
Sources
Related Entities
Related Claims
Unverifiedpass@k is a commonly used metric for measuring sampling quality in tasks such as code generation: k candidate answers are independently sampled for the same problem, and success is counted as long as any one of them passes verification.73% similarUnverifiedThe pass@k metric is widely adopted on standard benchmarks such as HumanEval and MBPP.69% similarVerifiedThe pass@k metric measures the probability that, when the model generates k candidate code snippets, at least one of them passes all test cases.65% similarUnverifiedIn evaluation set design, a holdout set refers to the portion of data deliberately kept out of the optimization process and used only for final validation, to detect overfitting, provided that the scorer definition reflects the good outputs you truly care about.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1093137API
curl https://kongchang.com/api/v1/knowledge/claims/1093137MCP
get_claim(id=1093137)