Unverified50% confidenceFactExact time
avg@12 means letting the model sample 12 times for the same batch of problems and taking the average of the correct rates on each problem, used to reduce the variance caused by sampling randomness.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedDue to non-determinism, the same Eval should be run 6 to 10 times to observe the distribution range of accuracy and calculate standard deviation and error.68% similarUnverifiedPrecision refers to the proportion of samples the model identifies as positive that are truly positive; recall refers to the proportion of all true positives that the model successfully identifies; the two typically have an inverse relationship.66% similarUnverifiedTypesafe's tests found that processing 13 questions in a single call is 12.2x cheaper and 10x faster than asking them one by one, with identical answers.65% similarVerifiedAUC equals the probability that, when one positive sample and one negative sample are randomly drawn from all positive and negative samples respectively, the predicted score of the positive sample is higher than that of the negative sample.65% similarUnverifiedIn imbalanced classification problems, a model can achieve very high accuracy even by predicting all samples as the majority class, so it is necessary to shift to more discriminative metrics such as precision-recall curves, F1 score, Matthews correlation coefficient, or AUC-ROC65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013307API
curl https://kongchang.com/api/v1/knowledge/claims/1013307MCP
get_claim(id=1013307)