Unverified50% confidenceOpinionExact time
Some argue that the 0% result depends heavily on the scoring threshold settings, since BLEU and cosine similarity penalize high-quality paraphrases harshly, and that the figure is better interpreted as no verbatim-level match rather than the models completely lacking knowledge of the content.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/3/2026
First Seen
Valid until: 2/1/2027
Sources
Related Entities
Related Claims
UnverifiedThe core of evaluation is scoring, typically expressing the quality of results using a continuous value from 0 to 1.73% similarUnverifiedIn LLM-as-a-judge evaluation, the judge's judgment is highly dependent on the wording of the prompt, and different descriptions of evaluation criteria can cause the pass rate to plummet from 100% to 0%73% similarUnverifiedRecording negative samples where 'a recommended solution did not apply' is just as important as recording successes; otherwise the model will fall into statistical bias, mistaking high-frequency solutions for high-accuracy ones71% similarUnverifiedThe core property of a strictly proper scoring rule is that the expected score is maximized if and only if the forecaster reports the probability they truly believe, with common examples including log-likelihood loss and the Brier score.70% similarUnverifiedIn A/B testing, accuracy differences of less than 2% are usually hard to distinguish, and a larger sample size is needed to reach reliable conclusions.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1130504API
curl https://kongchang.com/api/v1/knowledge/claims/1130504MCP
get_claim(id=1130504)