Unverified50% confidenceConceptExact time
Kendall's tau measures the agreement between two rankings and ranges from -1 to 1; with small samples the sampling fluctuation is large—for nine models (36 pairs), tau=0.61 has a standard error of about 0.12 and a 95% confidence interval of roughly 0.37 to 0.85; tau reflects only overall monotonic association and is insensitive to local swaps.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOn synthetic data with 10,000 samples, a 1% positive rate, and 495 negatives scoring 0.90 while the positives score 0.80 and 0.60 respectively, computing with a 0.5 threshold gives: ROC-AUC=0.9500, Average Precision=0.1449, precision=0.1681, recall=1.0000, accuracy=0.9505, TP=100, FP=495, TN=9405, FN=0.64% similarUnverifiedUnder a fixed compute budget, the optimal number of repetitions is N* = sqrt(a * sigma^2 / (b * tau^2)), where sigma^2 is the per-question variance, tau^2 is the per-repetition (resampling) variance, and a and b are cost coefficients.64% similarUnverifiedAs the model scales from 8B to 70B, the predictive power of fixed output vector similarity for behavior actually worsens, with a paired mean correlation change of -0.080 (95% confidence interval [-0.127, -0.035]).62% similarUnverifiedMatching BenchLM's top ten open-weight rows against Artificial Analysis by model name, nine models corresponded successfully (dots3-note Preview is missing from the latter's data); ranking these nine models separately gives a Kendall's tau of 0.61, with 29 concordant pairs and 7 discordant pairs.61% similarUnverifiedFor the 7B model, accuracy with the solver was 75.31% versus 72.02% for direct calculation, a paired improvement of +3.29 percentage points, with a 95% cluster interval of [-3.49, 10.38] crossing zero, making the advantage non-significant.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1069261API
curl https://kongchang.com/api/v1/knowledge/claims/1069261MCP
get_claim(id=1069261)