Unverified50% confidenceFactExact time
The GPQA Diamond subset has 198 questions in total, and at a 95% confidence interval the standard error of a single sample is about 2 percentage points, so a difference of 0.7 points is within the noise range.
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe accuracy data in the test has an error margin of approximately ±8–10 percentage points at n=100 sample size (based on a 95% confidence-level binomial confidence interval).71% similarUnverifiedOn a test set of 600 synthetic invoices, the QLoRA fine-tuned model achieved a full-invoice exact rate of 83.0%, an auto-approval rate of 86.8%, and 0 out of 461 validation-coverage field errors in approved invoices (upper bound 0.8%).71% similarUnverifiedIn the example, for the same set of human labels (18 pass, 2 fail), a lazy judge that passes everything gets an accuracy of 0.90 but a kappa of 0.00; a better judge gets an accuracy of 0.95 but a kappa of only 0.77, still not reaching the 0.80 threshold.70% similarUnverifiedLaya has an issue of high confidence when making errors, with instances of giving a confidence score of 0.952 despite a score of 0.70% similarUnverifiedThe relationship between the standard error of Monte Carlo estimation and the number of simulations N is that the standard error is proportional to 1/√N.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1049826API
curl https://kongchang.com/api/v1/knowledge/claims/1049826MCP
get_claim(id=1049826)