Unverified50% confidenceFactExact time
In RAG evaluation, the two rules 'any expected document in top-k' and 'all required evidence in top-k' may give completely opposite conclusions for multi-document queries, and the two are not interchangeable.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe quality ceiling of RAG depends on whether the retrieval stage can find the correct passages; if incorrect evidence is retrieved, the model will produce a plausible but wrong answer.69% similarUnverifiedRather than arguing about which metric has a higher average score, it is better to output the list of affected queries, including: which queries are judged differently under the two rules, the expected evidence for each query, the actual retrieval position, and the specific rule that causes the conclusion to flip.68% similarUnverifiedIn RAG scenarios, if contradictory information exists in the index, vector retrieval cannot automatically determine which piece of content is more credible and will include both in the generation context, leading to confused answers.66% similarUnverifiedIn document retrieval scenarios, different answers hit different document fragments making evidence distinguishable, but in code evaluation the evidence extracted from two functionally equivalent pieces of code overlaps heavily, causing validation to fail.66% similarUnverifiedThe evaluation set should cover several types of edge cases: rephrased questions, questions requiring combining two facts, queries containing spelling errors, and questions that no source can answer.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1048176API
curl https://kongchang.com/api/v1/knowledge/claims/1048176MCP
get_claim(id=1048176)