Unverified50% confidenceFactExact time
The study tested 13 models and found that even when models correctly computed the elapsed time, they still failed to apply this constraint to conversational recommendations in 62.1% of cases.
1
Sources
50%
Confidence
Long-term
Relevance
11/5/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn the six-person study sample, there were only about 23 moments where the tutor needed to decide 'what help to give', which is too small a sample to judge the quality of the model's decisions; by contrast, 123 unit tests can be written for the pure functions.75% similarUnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.74% similarUnverifiedThis test has limitations: only 14 groups, one 2-billion-parameter model, one run; three valid paraphrases were corrected before the run because they changed the legal meaning, and the corrections are in the same commit as the test set; the test set has not yet been run on larger models.73% similarUnverifiedThe research team conducted a retrospective ablation analysis across 12 models with a total of 121,680 valid trials, of which 79,853 failed the executability check.73% similarUnverifiedIn a verbatim-quotation benchmark spanning 13 models, 2 instruction variants, and 962 scored responses, not a single model corrected any planted error within a quotation.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1139002API
curl https://kongchang.com/api/v1/knowledge/claims/1139002MCP
get_claim(id=1139002)