Unverified50% confidenceBenchmarkExact time
Each model was run twice across all 24 cases, i.e., 48 episodes per model and 96 episodes in total, each time using a fresh conversation and simulator state with caching disabled.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedIn memory recall tests, the 12B model answered 82 out of 100 questions correctly and the 31B model answered 98 out of 100, with neither model hallucinating answers.60% similarUnverifiedAll four models correctly read all 48 clean invoices and 48 placebo invoices, with zero errors across 4,704 scoring responses, and 0 of the 521 fabricated numbers happened to equal the true total that was obscured.60% similarUnverifiedEach model was paired with four prompt types—zero-shot, few-shot, chain-of-thought, and few-shot plus chain-of-thought—each in two groups, producing approximately 52,000 labels in total.59% similarUnverifiedThe number of leads per run is configurable; the demo sets it to 5, but it can be increased to 25, 30, or even 100 per run in practice.58% similarUnverifiedIn the test, each statement was asked in isolation in a fresh conversation, while each family was also asked jointly in a single prompt; each model received approximately 160 calls with temperature set to 0.56% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1002789API
curl https://kongchang.com/api/v1/knowledge/claims/1002789MCP
get_claim(id=1002789)