Unverified50% confidenceFactExact time
In a real-world case, a coding agent's success rate dropped from 92% to 88% without any changes, which was a false positive rather than a true capability regression.
1
Sources
50%
Confidence
Long-term
Relevance
9/27/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe author deliberately introduced a genuine severe regression that reduced an agent's success rate from 92% to 52%, yet the statistical rule originally designed for this purpose passed the PR because the regression was concentrated in a small number of tasks rather than being uniformly distributed.69% similarUnverifiedUsing a prompt that simply says 'write less code' can reduce code volume by 33%, but the security metric drops to 95%.66% similarUnverifiedThis failure mode is highly consistent across seven models from four vendors, with the rate of being misled as high as 87% to 97%.66% similarUnverified文章声称约88%的程序员一开始会卡在不逐行审查代码的认知转变上65% similarUnverifiedIn tasks with propositional constraints, the proportion of models integrating candidates already verified as invalid is as high as 93%–100%.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/953018API
curl https://kongchang.com/api/v1/knowledge/claims/953018MCP
get_claim(id=953018)