Unverified50% confidenceFactExact time
The study validated the automated reference-free metrics using quality ratings from sighted raters and 7 BLV participants, finding a significant correlation between the two.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
Unverified数据标注评估考核中常见的质量指标包括标注者间一致性（IAA）和与黄金标准的偏差71% similarUnverified人工评估常用标注者间一致性（IAA）指标如Cohen's Kappa和Krippendorff's Alpha来确保评估质量可靠性，一般Kappa大于0.7为可接受水平70% similarUnverifiedThe core conclusion of the study is that agent evaluation should score closed-loop outcomes and report both reliability and coverage.70% similarVerifiedA valuable comparison testing tool should ideally have a standardized task set, an automated scoring mechanism, and result visualization capabilities.70% similarUnverifiedThe experiment used balanced accuracy as the evaluation metric, covering 8 classification datasets.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114625API
curl https://kongchang.com/api/v1/knowledge/claims/1114625MCP
get_claim(id=1114625)