Unverified50% confidenceFactExact time
Pass@k is a standardized metric for evaluating the code generation capability of language models, introduced by OpenAI in the HumanEval benchmark.
1
Sources
50%
Confidence
Long-term
Relevance
9/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOpenAI's Evals framework and EleutherAI's Language Model Evaluation Harness are commonly used open-source automated evaluation tools.69% similarUnverifiedHumanEval由OpenAI设计，包含164道代码补全题目，考察模型将自然语言转化为可运行代码的能力67% similarVerifiedHumanEval was proposed by OpenAI and contains 164 Python programming problems, primarily testing function-level code generation capability66% similarUnverified模型通过读取 OpenAPI 文档中的自然语言描述字段（summary、description）来理解每个接口的用途和触发条件65% similarUnverified该功能基于VS Code的LLM Language Tool API实现65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/931851API
curl https://kongchang.com/api/v1/knowledge/claims/931851MCP
get_claim(id=931851)