Unverified50% confidenceSolutionExact time
When multiple large language models strongly disagree in their annotations of the same document, it often indicates that the codebook rules at that point are ambiguous, equivocal, or insufficiently covered.
1
Sources
50%
Confidence
Long-term
Relevance
9/24/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedBecause the output of large language models is non-deterministic, the same input may produce different outputs, making it difficult to cover all edge cases with unit tests alone.79% similarUnverifiedThe semantic structure of code differs fundamentally from natural language, making it difficult for embedding models to accurately preserve information such as identifier naming, call relationships, and scope.78% similarVerifiedLarge language models have inherent non-determinism — even with identical prompt inputs, the model may return results that differ in format, content, or even logic at different times.78% similarUnverifiedWriting scoring rules into a prompt in natural language has drawbacks: wording variations across different calls may cause the model's understanding to drift, and the scoring logic cannot be directly parsed or reused by programs.76% similarVerifiedText generated by large language models tends to have low perplexity, with word choices that are overly consistent with probabilistic expectations, lacking the unexpectedness of human writing.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/946144API
curl https://kongchang.com/api/v1/knowledge/claims/946144MCP
get_claim(id=946144)