GPT-5.4 mini
A lightweight model in the GPT-5 series introduced by OpenAI, designed for cost-sensitive agentic application scenarios.
Timeline (last 90 days)
On the ring_push task, four models (Claude Haiku 4.5, Gemma 4 31B, GPT-5.4 mini, Gemini 3.7 Flash) completed the required changes in both tasks, yet lost old behaviors not restated in the prompt, passing fail-to-pass but failing pass-to-pass, and this failure reproduced stably across eight runs.
In the system migration S2 Q2 case, GPT-5.4 mini's explanation correctly preserved the geographic restriction that the North American window remained unchanged, yet returned the decision label as 'supported' instead of the correct 'contradicted'.
The free open-source model Gemma 4 answered all 60 answerable questions correctly and correctly labeled sources every time; gpt-oss-20b answered 59 correctly; the paid small models Claude Haiku 4.5 answered 36 correctly and GPT-5.4 mini answered 32 correctly.
On two-step reasoning questions, Claude Haiku 4.5 scored 6/30, GPT-5.4 mini scored 3/30, while Claude Sonnet 5.5, Gemini 3.8 Flash, and Gemma 4 all scored 30/30.
Claude Haiku 4.5 answered 'not found' in 21 of its 24 errors on two-step questions, while GPT-5.4 mini copied the literal wording of the referral as the answer in 20 of its 27 errors, and fabricated real-sounding names while citing actual documents in another 7.
In the segment where gains exceeded +30%, average model predictions ranged from 27.3% for Claude Opus 5.5 to 61.6% for GPT-5.4 mini, while the actual rise rate was 42.8%.
The only model among the 9 judged as "chasing momentum" was GPT-5.4 mini, with an extra uplift of +32.8 points (confidence interval +12.4 to +51.6) and a rank correlation with price gains of +0.73.
After hiding the price-gain figures from GPT-5.4 mini, its extra uplift dropped sharply from +32.8 points to +6.6 points, with the mechanism primarily being lower scores assigned to non-rising stocks rather than exceptional favoritism toward large gainers.
In the hard-to-verify category with injection bait, the number of questions for which GPT-5.4 mini judged incorrectly in both orderings rose from 3 to 39, Qwen3-Next 80B Instruct from 4 to 38, and GPT-5.4 mini and Qwen3-Next Instruct scored zero on hard questions (below the approximately 0.25 expected from random coin flipping).
In the hard-to-verify category with no bait, GPT-5.4 nano answered only 19/40 questions correctly in both orderings and had a 65% probability of selecting Response A; GPT-5.4 mini had a 66% probability of selecting A, Grok 4.20 non-reasoning was 62%, while strong judges were stable at 50%.
1 more timeline events
All Facts (12)
GPT-5.4及更早版本在提取文档中的数字时频繁出错，包括小数点位置偏移、千分位分隔符误读、负号遗漏等问题
80%UnverifiedOn the ring_push task, four models (Claude Haiku 4.5, Gemma 4 31B, GPT-5.4 mini, Gemini 3.7 Flash) completed the required changes in both tasks, yet lost old behaviors not restated in the prompt, passing fail-to-pass but failing pass-to-pass, and this failure reproduced stably across eight runs.
50%UnverifiedIn the system migration S2 Q2 case, GPT-5.4 mini's explanation correctly preserved the geographic restriction that the North American window remained unchanged, yet returned the decision label as 'supported' instead of the correct 'contradicted'.
50%UnverifiedOn two-step reasoning questions, Claude Haiku 4.5 scored 6/30, GPT-5.4 mini scored 3/30, while Claude Sonnet 5.5, Gemini 3.8 Flash, and Gemma 4 all scored 30/30.
50%UnverifiedClaude Haiku 4.5 answered 'not found' in 21 of its 24 errors on two-step questions, while GPT-5.4 mini copied the literal wording of the referral as the answer in 20 of its 27 errors, and fabricated real-sounding names while citing actual documents in another 7.
50%UnverifiedThe free open-source model Gemma 4 answered all 60 answerable questions correctly and correctly labeled sources every time; gpt-oss-20b answered 59 correctly; the paid small models Claude Haiku 4.5 answered 36 correctly and GPT-5.4 mini answered 32 correctly.
50%UnverifiedAfter hiding the price-gain figures from GPT-5.4 mini, its extra uplift dropped sharply from +32.8 points to +6.6 points, with the mechanism primarily being lower scores assigned to non-rising stocks rather than exceptional favoritism toward large gainers.
50%UnverifiedThe only model among the 9 judged as "chasing momentum" was GPT-5.4 mini, with an extra uplift of +32.8 points (confidence interval +12.4 to +51.6) and a rank correlation with price gains of +0.73.
50%UnverifiedIn the segment where gains exceeded +30%, average model predictions ranged from 27.3% for Claude Opus 5.5 to 61.6% for GPT-5.4 mini, while the actual rise rate was 42.8%.
50%UnverifiedIn the hard-to-verify category with no bait, GPT-5.4 nano answered only 19/40 questions correctly in both orderings and had a 65% probability of selecting Response A; GPT-5.4 mini had a 66% probability of selecting A, Grok 4.20 non-reasoning was 62%, while strong judges were stable at 50%.
50%UnverifiedIn the hard-to-verify category with injection bait, the number of questions for which GPT-5.4 mini judged incorrectly in both orderings rose from 3 to 39, Qwen3-Next 80B Instruct from 4 to 38, and GPT-5.4 mini and Qwen3-Next Instruct scored zero on hard questions (below the approximately 0.25 expected from random coin flipping).
50%UnverifiedGPT-5.4 mini got 5 of the 15 EMI and tax questions wrong, while scoring perfectly on the other three groups.
50%