Unverified50% confidenceBenchmarkExact time
Even in the hottest setting, at a 1% false-positive cutoff, 11% of rewrites were still flagged, and the AUC against the rewrite null hypothesis was still 0.67.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/14/2026
First Seen
Valid until: 1/12/2027
Sources
Related Entities
Related Claims
UnverifiedNone of the 9 models could clearly outperform the "fixed guess of 31%" baseline, with all models' AUC confidence intervals containing 0.5.75% similarUnverifiedCalibration means that when the model outputs 0.9, across a large number of similar samples about 90% are indeed the correct class; an uncalibrated model can have high AUC but its values cannot be used directly as probabilities.74% similarUnverifiedWhen the ratio of positive to negative samples is extreme, the area under the PR curve (AUPRC) is often more revealing of true performance than AUC.70% similarUnverifiedIn the BIRD benchmark, Aura's token-substitution approach achieved an accuracy of 67.6%, only 0.6 percentage points below the 68.2% achieved with real data, while the number of leaked personal information items dropped from 849 to 0.69% similarUnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1029975API
curl https://kongchang.com/api/v1/knowledge/claims/1029975MCP
get_claim(id=1029975)