Unverified50% confidenceBenchmarkExact time
When asked to provide reasons for a decision by pointing out the jurisdictional basis, the proportion of models citing the correct legal basis is as high as 66.7%–100%.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAfter the legal basis cited by the model is replaced with an irrelevant authority, the rate at which the decision changes is only 0.0%–21.7% on CaseHOLD, 30.0%–76.7% on ECHR and SCOTUS, and 43.3%–50.0% on ContractNLI.66% similarUnverifiedA rule was set in the experiment: any judgment with a confidence level below 60% would be directly handed over to human processing.66% similarUnverifiedIn providing correct, domain-specific justifications, performance across different models ranged from 25.6% all the way to 76.9%.65% similarVerifiedModel calibration is defined in statistics as: if a model claims a certain answer has 70% confidence, then the actual proportion of correct answers among a large number of such claims should be close to 70%.65% similarUnverifiedWhen a model answered incorrectly and the user stated the correct answer, 77.4% of the 115 cases accepted the correction.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013276API
curl https://kongchang.com/api/v1/knowledge/claims/1013276MCP
get_claim(id=1013276)