Unverified50% confidenceOpinionExact time
The VTF900 study shows that high AUC under small-scale galleries does not translate into reliable performance at deployment scale, and model rankings derived from hundred-person-scale galleries may undergo substantial reordering at the 900-person scale.
1
Sources
50%
Confidence
Long-term
Relevance
10/20/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAt the full VTF900 gallery scale (N=900) under frontal conditions, the ViT-B model achieves 97.68% AUC and 51.10% TAR@1%FAR.69% similarUnverifiedCalibration means that when the model outputs 0.9, across a large number of similar samples about 90% are indeed the correct class; an uncalibrated model can have high AUC but its values cannot be used directly as probabilities.66% similarUnverifiedOn synthetic data with 10,000 samples, a 1% positive rate, and 495 negatives scoring 0.90 while the positives score 0.80 and 0.60 respectively, computing with a 0.5 threshold gives: ROC-AUC=0.9500, Average Precision=0.1449, precision=0.1681, recall=1.0000, accuracy=0.9505, TP=100, FP=495, TN=9405, FN=0.65% similarUnverifiedEven in large models ranging from 70B to 72B, the penetration rate of safety instructions remains below 10%, and increased scale does not make safety prompts more effective.64% similarUnverifiediPSAE评分对Binder二分类预测的AUC仅为0.65，远低于可靠分类器水平63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1074836API
curl https://kongchang.com/api/v1/knowledge/claims/1074836MCP
get_claim(id=1074836)