Unverified50% confidenceEventExact time
While generating the OSWorld 2.1 reference runs, Simular had tasks stop after one step due to the model key's usage limit, yet they were recorded as successful with a score of 0, and the average score over the whole folder was computed as 0.40.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedOSWorld maintainers have an unmerged fix targeting 14 of the 108 tasks, whose scorers turn a failed call to the judge model into a score of 0.73% similarUnverifiedOSWorld 2.0中模型评估仅占总得分的11.53%，且没有任何任务的模型评估占比超过50%69% similarUnverifiedThe 82 tasks in OS World were not run under identical conditions—half of the tasks retained the original baseline scores, and the invested resources and number of attempts also differed—so the 7-percentage-point improvement cannot be entirely attributed to memory.65% similarUnverifiedIn the experiment, the task_score under the shaped reward configuration was only 0.125, compared to 0.417 for the naive reward, with an effect size of d=4.47.62% similarUnverifiedIn a semantic segmentation task, the baseline model UNet achieved an original mIoU score of 0.817.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1009488API
curl https://kongchang.com/api/v1/knowledge/claims/1009488MCP
get_claim(id=1009488)