Unverified50% confidenceFactExact time
Differential item functioning (DIF) analysis finds that models from different developers show systematic differences in HarmBench scores when their "refusal ability" is the same.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
UnverifiedOn the strongest models, the differences between experimental groups fall within confidence intervals, and researchers characterized the results as directional patterns and soundness properties rather than proven performance lower bounds67% similarUnverifiedA psychometric audit study of the HELM Safety benchmark points out that similar total scores between models may mask entirely different attribute profiles.67% similarUnverifiedbenchmark分数与实际开发体验之间存在显著落差，模型可能在标准测试中领先却在真实项目中表现平平66% similarUnverifiedDiscrepancies among prior EF prediction studies largely stem from inconsistencies in target populations, as different studies adopt heterogeneous inclusion criteria and different definitions of extubation failure.65% similarUnverified不同模型在基准测试上的性能差异往往远小于同一模型在不同任务类型、不同输入格式下的行为差异65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013427API
curl https://kongchang.com/api/v1/knowledge/claims/1013427MCP
get_claim(id=1013427)