Unverified50% confidenceBenchmarkExact time
Of the 962 responses, 51 failed, distributed as 26 truncations, 13 boundary errors, 12 paraphrases, and 0 corrections.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAmong the 91 failure cases, 51 were at the API layer and 25 were at the wrapper layer, accounting for the vast majority.69% similarUnverifiedAmong trajectories that terminated cleanly yet actually failed, field value errors accounted for 77.61%, unexpected additional side effects for 43.30%, and missing required side effects for 25.36% (categories overlap).68% similarUnverifiedIn the experiments, the task resolution rate was 17% for zero-shot generation, 27% for generic self-correction, 23% for error-only feedback, and 90% for diagnostic counterexample feedback (A-CEGIS).65% similarUnverifiedAmong the 35 genuine failure cases, only 3 did not involve any assertions, indicating that the vast majority of failures stem from accepting biased statements rather than from missing information.65% similarUnverifiedAll four models correctly read all 48 clean invoices and 48 placebo invoices, with zero errors across 4,704 scoring responses, and 0 of the 521 fabricated numbers happened to equal the true total that was obscured.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1000251API
curl https://kongchang.com/api/v1/knowledge/claims/1000251MCP
get_claim(id=1000251)