Unverified50% confidenceBenchmarkExact time
In introspection-style deception tests, the probe needs to distinguish conversation records containing the model's true hidden objective from records containing other objectives, and the probe's AUC reaches as high as 99.7%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
UnverifiedAUC is the standard metric for measuring the performance of a binary classifier, with values from 0.5 to 1.0, and 98.8% AUC means that in about 99% of sample pairs, the probe can correctly rank conversations containing deception ahead of conversations without deception.78% similarUnverifiedAUC将所有阈值下的表现等权平均，掩盖了在特定低误报率工作阈值下的真实表现，在高安全场景可能给出误导性评估70% similarUnverifiedA learner, when detecting forged receipts on this dataset, found that regardless of how the model was adjusted, the ROC-AUC consistently hovered between 0.46 and 0.54, close to random guessing.67% similarUnverified实验采用8:2比例划分数据，20%作为内部测试集，训练集内部使用5折交叉验证，并移除重复记录以避免数据泄漏67% similarUnverifiedIn a facial recognition system with 99.9% accuracy, if there are only 10 real targets among 1 million people, approximately 99% of the roughly 1,010 alerts generated would be false positives.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013466API
curl https://kongchang.com/api/v1/knowledge/claims/1013466MCP
get_claim(id=1013466)