Unverified50% confidenceTradeoffExact time
Because the experiment used temperature=0 and a fixed seed in a single deterministic run, any field flip moves the score by about 2.8 percentage points out of 12 tasks × 3 fields = 36 units, so the gap between 91.7% and 83.3% cannot be directly generalized into a statistically meaningful general advantage.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedEven under temperature 0 conditions, 0.07% to 10.2% of rows across five executions still change their operator output, with the exact proportion depending on the model77% similarUnverifiedFor two-step and three-step compositional tasks, the median paired difference in accuracy under the masking condition reaches 0.846 and 0.859, respectively.72% similarUnverifiedFor the 7B model, accuracy with the solver was 75.31% versus 72.02% for direct calculation, a paired improvement of +3.29 percentage points, with a 95% cluster interval of [-3.49, 10.38] crossing zero, making the advantage non-significant.72% similarUnverifiedIn the specific experimental case, the overlapping-anchor scheme (n=912) showed the model at 55.2% versus the majority class at 51.1%, a net advantage of +4.1pp, about 2.4 standard errors; the non-overlapping-anchor scheme (n=163) showed 59.5% versus 54.0%, a net advantage of +5.5pp, about 1.4 standard errors.71% similarUnverifiedOn one task, the proportion of rows that change operator output under temperature 0 reached as high as 24.0%70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1025952API
curl https://kongchang.com/api/v1/knowledge/claims/1025952MCP
get_claim(id=1025952)