Unverified50% confidenceOpinionExact time
Memory benchmark scores can fluctuate enormously depending on the evaluation protocol, and the same system may differ by more than ten percentage points under a different evaluation process.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAccording to the ForgetEval research, when leading memory systems were scored on test cases written by people other than the authors themselves, internal test scores of 63%-68% plummeted to 28%-33%.75% similarUnverifiedThe research team pointed out that the performance gains from inference-time context management in benchmarks exceeded most system-to-system differences that have been reported.74% similarUnverifiedIn many evaluations conducted on small datasets, scores fluctuate by several percentage points across different runs.74% similarUnverifiedWhen researchers run different testing frameworks using the same model, scores often fluctuate noticeably.74% similarUnverifiedResearch shows that in certain tasks, simply changing '请判断' to '请分析' in a prompt can cause accuracy fluctuations of 5–10 percentage points.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1030820API
curl https://kongchang.com/api/v1/knowledge/claims/1030820MCP
get_claim(id=1030820)