Unverified50% confidenceFactExact time
In the retrospective check, only 1 of 15 released revision units passed the release check.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
Related Entities
Related Claims
UnverifiedThe team audited the formulas in the benchmark against current clinical guidelines one by one, flagging 16 entries among the 55 calculators that had issues with versioning, intended use, or coefficients.62% similarUnverified7 out of the 13 evaluated VLMs passed the surrogate test under at least one configuration.61% similarUnverified15 scenarios where UK and US medical guidelines diverged were excluded from the test to ensure there was no dispute over which cases required medical attention.60% similarUnverifiedThe study's authors acknowledge that the development set of 18 documents and 18 queries is very small, with limited statistical power, and the conclusions require independent verification.59% similarUnverifiedThese benchmark results are currently self-reported from a single source, the evaluation sets are limited in size (30 HumanEval questions, a 502-question MMLU subset), and they have not yet been independently reproduced and cross-validated by the community.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1133817API
curl https://kongchang.com/api/v1/knowledge/claims/1133817MCP
get_claim(id=1133817)