Claude Sonnet 5

Sonnet 5.5 runs more than 30% faster than Sonnet 5.

Claude Sonnet 4 is a new-generation large language model released by Anthropic, belonging to the mid-range product line of the Claude 4 series.

The test covered four vision models: Claude Sonnet 5 (Anthropic), Gemini 3 Flash (Google), Gemini 3.1 Flash-Lite (Google), and GPT-5.4 nano (OpenAI).

The Hermes Agent configuration process is: run `sudo hermes setup`, choose Full Setup, choose the AI model provider (the author chose Anthropic's Claude Sonnet 5) and paste the API Key.

The researchers queried Kaggle's 46-model catalog, but the first expansion round exhausted the daily AI quota, and the final paired analysis included 7 endpoints that completed all 90 image prompts: Claude Haiku 4.5/5.5, Claude Sonnet 5/5.5, Gemini 3.5 Flash-Lite, Gemini 3.7 Flash, GPT-5.4 Mini.

Claude Haiku 5.5 scored 0.95, surpassing Claude Sonnet 5's score of 0.88.

Frontier reasoning models can reverse an opponent's adversarial bad suggestions into offensive weapons, while smaller models do not possess this counter-exploitation capability.

Claude Sonnet 5 had a rock-paper-scissors win rate of only 33.3% under the neutral control condition, but its win rate surged to 86.1% under persuasion and deception conditions (31 wins and 5 draws out of 36 rounds).

On two-step reasoning questions, Claude Haiku 4.5 scored 6/30, GPT-5.4 mini scored 3/30, while Claude Sonnet 5.5, Gemini 3.8 Flash, and Gemma 4 all scored 30/30.

Claude Sonnet 5 made wild guesses on 25% of cold cases during the development phase, and as many as 30% on Kaggle.

With the rule book, Haiku and Sonnet achieved identical scores, but Haiku is 14 times cheaper.

The eight models tested include GPT-6 Luna, Claude Haiku 5.5, Gemini 3.5 Flash-Lite, Qwen 3 Next 80B, Gemma 4 31B, GPT-6 Sol, Claude Sonnet 5, and Gemini 3.7 Flash, with each model running 107 investigations per track.

In the test, the top tier (Overall 83.33) is occupied by Gemini 3.5 Flash, Claude Sonnet 5, and Claude Opus 5.

Claude Haiku 4.5 had a 20.2% capitulation rate under the first incorrect pressure but only a 7.6% rate of being broken by repetition; Claude Sonnet 5 resisted 94.4% of the first pressure attempts but lost 15.7% of those under repetition.

The median confidence difference across models when fabricating totals is enormous: Gemini 3 Flash and Flash-Lite at 95, GPT-5.4 nano at 62, and Claude Sonnet 5 at only 20.

When the total was obscured but each line item was visible, Claude and Gemini 3 Flash got all 48 correct, with all failures coming from the smaller models (Flash-Lite failed 31%, nano failed 79%), and nano got only 12/48 correct even when explicitly instructed to add up the items.

Fabrication rates when the total is fully obscured: Claude Sonnet 5 at 38%, Gemini 3 Flash at 21%, Gemini 3.1 Flash-Lite at 44%, GPT-5.4 nano at 73%; fabrication rates when the total is half-obscured: Claude at 92%, the other three models all at 100%.

When asked directly whether the total was visible, Claude, Gemini 3 Flash, and Flash-Lite answered across all 48 receipts that the total was not visible and could not be calculated, yet when asked directly for the total, they still fabricated one 21–44% of the time.

The client-side auto-mode classifier now ignores ANTHROPIC_DEFAULT_SONNET_MODEL values pinned to Claude Sonnet 5.5 or Opus 5.5, and uses Claude Sonnet 5 instead.

A spot-check reproduction using Claude Sonnet 5 based on full-replay control reproduced the pattern of embedding retrieval being unstable while structured storage remained stable.