Expired50% confidenceFactTime unknown
Claude 3.7 Thinking was used as the LLM judge in the evaluation, providing high consistency with minimal variance across repeated runs
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
Deep Dive Review of AI Coding Assistants: Copilot at the Bottom — Who's the Real King?
bilibili比特光锥_BightCone
Related Claims
UnverifiedThe evaluation uses three scoring dimensions: Instruction Following, Unit Testing, and LLM as Judge63% similarUnverified评测设置了语义正确率、三次全对率和格式正确率三个维度指标59% similarUnverifiedFrontier reasoning LLMs (such as Claude 3.5 Sonnet) can follow approximately 150–200 instructions with relatively high consistency.59% similarUnverifiedClaude Opus 4.7 at an intermediate checkpoint achieves a pass rate of 57%, yet its final accuracy is only 39.8%58% similarUnverifiedGLM4 scored 74.4% in code reasoning and general knowledge tests, ranking second only to Claude 3 Opus.56% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/59565API
curl https://kongchang.com/api/v1/knowledge/claims/59565MCP
get_claim(id=59565)