Unverified50% confidenceEventExact time
Among the six safety questions in MT-Bench-PT, on two questions the LLM judge refused to give a score no matter what the evaluated answer was, silently reducing the effective sample size N under that category.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe reason LLM-as-Judge is effective is that evaluation is a less difficult task than generation; the judge LLM only needs to make judgments on constrained scoring dimensions, making its results more stable than generation.69% similarUnverifiedExperiments found that generic stopword lists fell below the 'no removal' baseline in every single test, actually dragging down classification performance.67% similarUnverifiedLLM evaluation requires finding a balance between tests that are too loose (letting real issues pass) and tests that are too strict (generating large numbers of false positives), and thresholds often need to be calibrated through experimentation.66% similarUnverifiedIn zero-shot classification, the scores of all candidates sum to 1; the fewer the candidates, the larger the probability space available to each, so the threshold parameter needs to be calibrated individually based on the specific number of categories.66% similarVerifiedThe fundamental reason prompt injection is difficult to defend against is that LLMs cannot reliably distinguish between data and instructions.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1032800API
curl https://kongchang.com/api/v1/knowledge/claims/1032800MCP
get_claim(id=1032800)