Unverified50% confidenceFactExact time
The test designed 50 automatically verifiable tasks, divided into five categories: multi-step math computation, extracting JSON from order notes, classifying support tickets, conditional filtering and sorting, and strict-format writing, with 10 questions per category, each with a standard answer.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedBuilding a small test set should start with 30 to 50 real questions, each accompanied by the correct answer and the document containing the answer, measuring retrieval (whether the correct text chunk appears in the top results) and generation (whether the final answer is supported by the retrieved text) separately.75% similarUnverifiedThe test questions are divided into 40 easy-to-verify questions and 40 hard-to-verify questions, and all standard answers of the math, code, and date types are validated in Python.74% similarUnverifiedMost traditional LLM benchmarks focus on relatively closed tasks such as single-turn question answering, code generation, and reasoning problems.73% similarUnverifiedThe team wrote automated acceptance scripts covering 63 test cases—including key player queries, factual trap questions, sensitive topic interception, and multi-turn conversations—all of which passed, with one-click regression testing supported.73% similarUnverifiedThe benchmark uses a unified set of 100 questions, with only one search issued per question and the same answering model used across all configurations to process the returned content.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1018317API
curl https://kongchang.com/api/v1/knowledge/claims/1018317MCP
get_claim(id=1018317)