Unverified50% confidenceFactExact time
The original buggy version of the billing function used in testing passed only 138 of 215 tests, while the fixed version passes all 215 (15 hand-written edge cases plus 200 random cases).
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/19/2026
First Seen
Valid until: 11/2/2026
Sources
Related Entities
Related Claims
UnverifiedThis benchmark uses 10 original short documents of 200 to 250 words each, with 18 planted anomalies, 37 scoring entries in total (18 dirty and 19 clean), testing 13 models with 74 prompts per model.72% similarUnverifiedThis test has limitations: only 14 groups, one 2-billion-parameter model, one run; three valid paraphrases were corrected before the run because they changed the legal meaning, and the corrections are in the same commit as the test set; the test set has not yet been run on larger models.68% similarUnverifiedRetesting with the same 20 files, 18 compiled directly; after one more iteration letting the sessions receive compiler error messages, it reached 19.68% similarUnverifiedLargha-150M-v1's training mixture includes a pool of 500 error corrections, which were mined from the model's predictions on typed-decision test rows, so 0.906 is a fitting reading rather than a zero-shot result.67% similarUnverifiedIn CLM experiments, the accuracy of selecting the correct tool from 8 tools was 86%, but dropped sharply to 17% when selecting from all 1,080 tools, with the main error being confusion between tools with similar names; TiFF is limited to a maximum of 255 options.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1067184API
curl https://kongchang.com/api/v1/knowledge/claims/1067184MCP
get_claim(id=1067184)