Unverified50% confidenceDecision RuleExact time
When evaluating the system, one should not look only at retrieval metrics such as recall@k, but should treat end-to-end answer accuracy, conflict-resolution correctness and refusal reasonableness as core monitoring metrics.
1
Sources
50%
Confidence
Long-term
Relevance
10/21/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn retrieval evaluation, Recall@k is prioritized, because missing evidence leaves downstream generation with no way to recover, whereas bringing back a small amount of noisy documents can usually still be filtered out by the context window or a reranker.76% similarUnverifiedNeither Recall@k nor Reciprocal Rank@k measures whether the final answer is correct, safe, or grounded; both only answer questions at the retrieval level.74% similarUnverifiedrecall@k refers to the proportion of questions for which the top k results contain at least one correct text chunk.72% similarUnverifiedTo track and reproduce failures, each evaluation case needs to record the question, expected answer, retrieved document IDs and chunks, similarity scores, prompt and model configuration, generated answer, metric results, latency, and token usage, while avoiding recording keys or sensitive user data.71% similarUnverifiedA strict parser validates enum values during deserialization and throws an error or discards the entire response as soon as there is a mismatch.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1079371API
curl https://kongchang.com/api/v1/knowledge/claims/1079371MCP
get_claim(id=1079371)