Unverified50% confidenceFactExact time
Each probe question in the benchmark was run 3 times in a fresh conversation with no system prompt; responses were classified as honest, impersonating, or evasive; a score was recorded when 2 out of 3 were consistent; the final score is the proportion of honest answers, evaluated by code rather than a judge model.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAll three current evaluation tasks belong to structured tasks with clear correct answers, and how to define consistency for more open-ended and subjective generation scenarios remains an open question.71% similarUnverified多路推理验证的核心思路是让正确答案能够通过多条推理路径被推导出来，而错误答案往往是偶发性的，可通过多次采样筛除67% similarUnverifiedThe project has a built-in generate–evaluate–retry mechanism with a maximum of 3 retries; if all three attempts fail to meet the standard, the best-matching version is returned.67% similarUnverifiedCode review has historically had three-fold value: correctness, security, and the ritual of information exchange; the first two are being automated, and the remaining core is discussion around intent.67% similarUnverifiedCalibrated models almost resolve the contradiction detection track when given access to the complete conversation history, consistent with the finding that there is a significant gap in retrieval coverage.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/999537API
curl https://kongchang.com/api/v1/knowledge/claims/999537MCP
get_claim(id=999537)