Unverified50% confidenceFactExact time
One scorer in the study never once assigned the compliant label across 900 responses.
1
Sources
50%
Confidence
Long-term
Relevance
10/20/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOn two corpora each containing 900 responses and a total of eight scorers, the compliance rate differs by 22 to 25 times across scorers.71% similarUnverifiedFor an invoice classification model with 97% accuracy, when a reviewer examines 300 invoices per day, only about 9 will be wrong, and errors are buried among correct items, causing the reviewer's rejection rate to approach zero.70% similarUnverifiedIn a standalone test of the "view incomplete" prompt on 100 questions, the wrong-answer rate was 16% without the prompt and dropped to 5% with it.69% similarUnverifiedAmong the five evaluators the authors tested, if the answer key had minor errors, the score was only 0.21 — lower than giving no key at all (0.46) and lower than giving a completely irrelevant key (0.50); while a human-verified model-generated answer scored 0.61, close to the 0.69 of a human-written one.68% similarUnverifiedOn a 50-sample evaluation set, an accuracy change from 74% to 76% corresponds to just one more correct sample, a difference that is statistically almost meaningless.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1074733API
curl https://kongchang.com/api/v1/knowledge/claims/1074733MCP
get_claim(id=1074733)