Unverified50% confidenceEventExact time
The author has open-sourced the full experimental code and raw judge outputs, and plans to rewrite the 13 single-sentence reference answers as multiple claims and re-judge, to verify whether this reduces judge noise on recall.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/19/2026
First Seen
Valid until: 11/2/2026
Sources
Related Entities
Related Claims
UnverifiedThe rebuttal phase allows authors to submit a brief response after seeing the initial review comments, to clarify misunderstandings or supplement experiments.70% similarUnverifiedAt the implementation level, the authors have created a shared baseline and answer structures for twelve cases, implemented scoring and validation code, and added automated tests, with the latest development run passing all ten tests.69% similarUnverifiedThe author recommends freezing the question list before scoring and reporting every row's result and the reason for any missing ones; where each question can be annotated with the documents that should be retrieved, compute recall@k with simple counting, since under deterministic retrieval such metrics carry no run-to-run noise at all; and write reference answers as multiple checkable claims, to avoid single-sentence references turning recall into a coin flip.68% similarUnverifiedThe author built a simulated test framework (held-out test), implanting known text on a fake plaque and then cracking it with the same method to measure the reliability of the method.68% similarUnverifiedThe author fixed the benchmark by switching to serial calls, adding retries, and printing "answered: X of 60" before reporting the score, in order to ensure the honesty of benchmark scores.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065200API
curl https://kongchang.com/api/v1/knowledge/claims/1065200MCP
get_claim(id=1065200)