Unverified50% confidenceBenchmarkExact time
In tests on ten agents, the two review pipelines achieved a checkpoint agreement of κ = 0.71 and an agent ranking consistency of Kendall τ = 0.87.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/28/2026
First Seen
Valid until: 1/26/2027
Sources
Related Entities
Related Claims
UnverifiedThe experiment used balanced accuracy as the evaluation metric, covering 8 classification datasets.69% similarUnverifiedPAWS is validated on 2,522 stratified-sampled behavior samples, with independent AI reviewers and human reviewers achieving an initial agreement rate of 89.4% on interaction pattern judgments.69% similarUnverifiedAdversarial test cases should be written as deterministic binary pass/fail checkers rather than using fuzzy similarity scores.68% similarUnverifiedThe tau-bench benchmark focuses on whether an agent reaches the expected database state and requires consistent success across multiple repeated attempts.68% similarUnverified人工评估常用标注者间一致性（IAA）指标如Cohen's Kappa和Krippendorff's Alpha来确保评估质量可靠性，一般Kappa大于0.7为可接受水平67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1106109API
curl https://kongchang.com/api/v1/knowledge/claims/1106109MCP
get_claim(id=1106109)