AUC
A core aggregate evaluation metric for binary classification models, with values ranging from 0.5 to 1.0. AUC is robust to class imbalance and measures a model's discriminative ability across all classification thresholds. Values of 0.8–0.9 are considered good; values above 0.9 are considered excellent.
Core Facts
AUC值域为0.5到1.0，0.5代表随机猜测、1.0代表完美预测，不受样本类别不平衡影响
According to common industry standards, AUC > 0.9 is excellent, 0.8–0.9 is good, 0.7–0.8 is fair, 0.5–0.7 is poor, and 0.5 is equivalent to random guessing.
An AUC=1 on the ROC curve indicates a perfect classifier, while AUC=0.5 is equivalent to random guessing.
Timeline (last 90 days)
The z-score is the output statistic of the SynthID detector, measuring how far the bias of watermark tokens in the tested text deviates relative to unbiased random text; AUC measures the detector's overall ability to distinguish watermarked text from clean text, with 0.5 indicating no discrimination at all and 1.0 indicating perfect discrimination.
An AUC below 0.5 after swapping names and definitions indicates that the model scores can still distinguish the classes but the direction is systematically reversed, which cannot be corrected by adjusting the threshold.
On in-the-wild datasets, the AUC (area under the curve) of traditional lip-sync detectors often drops significantly, because the lip region is blurred or invisible in many forged clips, whereas physically aware detectors can leverage the independent cue of distance-speech energy consistency to maintain high discriminative ability.
AUC is a metric measuring the ranking ability of a binary classification model, ranging from 0.5 to 1, and does not depend on a specific threshold; the McNemar test is a statistical test for paired binary classification data, with p<0.05 generally considered significant.
AUC is the standard metric for measuring the performance of a binary classifier, with values from 0.5 to 1.0, and 98.8% AUC means that in about 99% of sample pairs, the probe can correctly rank conversations containing deception ahead of conversations without deception.
In the full-data comparison, relative to the strongest baseline, LLM-BlockFE's absolute AUC improvement on each dataset is between 0.0069 and 0.0358.
In mature risk control models, an AUC improvement of 0.01 often means a substantial change in the bad debt rate or approval rate.
In credit risk scenarios, if training samples include repayment records updated after the application occurred, the model's AUC may be artificially inflated by 5 to 10 percentage points, with performance dropping sharply after deployment
When the control sentences were replaced with sentences of a different register from the member sentences, the AUC of the MIA detector soared from 0.83 to 0.94.
Model evaluation should distinguish between offline metrics (AUC, precision/recall) and online metrics (A/B testing, business KPIs).
5 more timeline events
All Facts (20)
AUC值域为0.5到1.0，0.5代表随机猜测、1.0代表完美预测，不受样本类别不平衡影响
75%VerifiedAccording to common industry standards, AUC > 0.9 is excellent, 0.8–0.9 is good, 0.7–0.8 is fair, 0.5–0.7 is poor, and 0.5 is equivalent to random guessing.
70%VerifiedAn AUC=1 on the ROC curve indicates a perfect classifier, while AUC=0.5 is equivalent to random guessing.
70%VerifiedAUC equals the probability that, when one positive sample and one negative sample are randomly drawn from all positive and negative samples respectively, the predicted score of the positive sample is higher than that of the negative sample.
65%UnverifiedAUC可以通过梯形法则对ROC曲线下方面积进行数值积分得到，也可以通过Mann-Whitney U统计量等价计算
95%UnverifiedAUC的取值范围在0到1之间，数值越大代表模型分类能力越强
95%Unverified识别过拟合的经典方法是比较训练集AUC和验证集AUC——如果训练集AUC接近1.0而验证集AUC明显偏低，就是过拟合的典型信号
90%UnverifiedAUC（Area Under Curve）是衡量分类模型好坏最常用的指标之一
85%Unverified利用正样本秩次之和快速计算AUC的算法时间复杂度为O(n log n)
60%UnverifiedThe z-score is the output statistic of the SynthID detector, measuring how far the bias of watermark tokens in the tested text deviates relative to unbiased random text; AUC measures the detector's overall ability to distinguish watermarked text from clean text, with 0.5 indicating no discrimination at all and 1.0 indicating perfect discrimination.
50%UnverifiedAn AUC below 0.5 after swapping names and definitions indicates that the model scores can still distinguish the classes but the direction is systematically reversed, which cannot be corrected by adjusting the threshold.
50%UnverifiedOn in-the-wild datasets, the AUC (area under the curve) of traditional lip-sync detectors often drops significantly, because the lip region is blurred or invisible in many forged clips, whereas physically aware detectors can leverage the independent cue of distance-speech energy consistency to maintain high discriminative ability.
50%UnverifiedAUC is a metric measuring the ranking ability of a binary classification model, ranging from 0.5 to 1, and does not depend on a specific threshold; the McNemar test is a statistical test for paired binary classification data, with p<0.05 generally considered significant.
50%UnverifiedAUC is the standard metric for measuring the performance of a binary classifier, with values from 0.5 to 1.0, and 98.8% AUC means that in about 99% of sample pairs, the probe can correctly rank conversations containing deception ahead of conversations without deception.
50%UnverifiedIn the full-data comparison, relative to the strongest baseline, LLM-BlockFE's absolute AUC improvement on each dataset is between 0.0069 and 0.0358.
50%UnverifiedIn mature risk control models, an AUC improvement of 0.01 often means a substantial change in the bad debt rate or approval rate.
50%UnverifiedIn credit risk scenarios, if training samples include repayment records updated after the application occurred, the model's AUC may be artificially inflated by 5 to 10 percentage points, with performance dropping sharply after deployment
50%UnverifiedWhen the control sentences were replaced with sentences of a different register from the member sentences, the AUC of the MIA detector soared from 0.83 to 0.94.
50%UnverifiedModel evaluation should distinguish between offline metrics (AUC, precision/recall) and online metrics (A/B testing, business KPIs).
50%Unverified在深度伪造检测的生产环境中真实图像与伪造图像比例可能高达1000:1甚至更高，极端不平衡下AUC对长尾失败几乎没有区分度
50%