Did Opus 5.5 Get Quietly Nerfed? Developer Reproduces Side-by-Side Comparison With Identical Prompts

A developer posted a set of compelling — and controversial — evidence on Reddit: using the exact same prompts and reference images, he generated the same Godot game project with Claude Opus 5.5 on its launch day versus roughly ten days later, and the latter result was noticeably worse. The post reignited a long-running community debate about AI models being quietly nerfed or degraded after release.

A Reproduction Experiment With Receipts

The developer's test subject carries a personal touch — he wanted to faithfully recreate David Braben's classic game Lander using Opus 5.5, the first game he played as a child at a UK school, running on an Acorn Archimedes with an early ARM chip.

His experimental design is worth noting: the Godot repository generated on launch day (September 23rd) — targeting macOS, Metal rendering, C# — was saved separately on disk and never used as a reference for subsequent runs. On October 2nd, he spun up a completely fresh, from-scratch workspace and fed in the exact same prompts and two original reference images. Both runs had Opus's effort level set to Extra High.

In other words, he deliberately controlled for variables in an attempt to rule out "non-determinism" — the most common counter-argument.

The Specific Regressions the Author Identified

According to the post, the later version showed multiple observable regressions compared to the launch-day version:

• Rendering glitches: Scene props "stutter and twitch" relative to world space when the camera moves — an issue absent in the launch-day version.
• Missing physics: The original implemented ship break-apart physics, which wasn't explicitly requested in the prompt — a bonus the model added on its own. The new version has none.
• Worse UX: No introductory control menu, just a small line of text permanently overlaid on screen.
• Visual degradation: The launch pad turrets look noticeably worse.
• Broken shooting logic: Bullets render incorrectly when the ship is moving, and they spawn from the ship's tail and no-clip through it.
• Missing camera switching functionality.

Root Cause Analysis at the Code Level

This post is more persuasive than the typical "it just feels worse" complaint because the author actually read the generated code.

He found that the degraded version's Godot project had physics interpolation enabled globally, while Props.cs rebuilds the props list (trees, etc.) from scratch on every frame for every tile step — renumbering each object's slot and resetting the object count every frame.

The author states bluntly that anyone with basic knowledge of Godot's documentation would avoid this mistake. This means the problem isn't just "worse performance" — the engineering quality of the generated code itself has regressed, and he worries that as the project grows, this kind of foundational messiness will compound into increasingly severe bugs.

Physics interpolation is a technique in game engines used to smooth out the difference between render frames and physics update frames. Godot's physics system runs at a fixed rate (60Hz by default), while the render framerate may be higher; interpolation estimates object positions between physics frames to make motion appear smoother. However, if physics interpolation is enabled globally without properly handling object lifecycles — such as Props.cs destroying and rebuilding the props list every frame — the engine will attempt to interpolate objects that have been flagged as "newly spawned," since they have no previous-frame position reference. They'll "pop" from the origin or an incorrect position to their current position, causing the visual stuttering and twitching the author describes. The correct approach is either to build the props list once during scene initialization and update it only as needed, or to explicitly call reset_physics_interpolation() on dynamically spawned objects to tell the engine to skip interpolation for the first frame. The generated code committed both errors simultaneously — unnecessary global enablement and incorrect object management — and their combination produced the hard-to-track frame-level bugs.

Can "Non-Determinism" Explain All of This?

LLM outputs are inherently non-deterministic — a challenge any such comparison inevitably faces. The author addresses this directly: variation between runs with the same prompts and reference images is usually small, and what he got was a noticeably worse result that he doesn't believe can be attributed to randomness.

This is the core point of contention. Supporters will argue that observable, reproducible code-level regressions constitute meaningful evidence; skeptics will point out that a single comparison is far too small a sample, and that temperature sampling, subtle context differences, or simply bad luck can produce wildly different results in a one-shot generation. Strictly speaking, proving a model has been degraded requires repeated experiments with statistical analysis — not a single data point.

A language model's "temperature" parameter controls the randomness of its output: at temperature 0 the model produces the most deterministic output, while higher temperatures yield more creative but less stable results. Code generation tasks typically use lower temperatures, but even then, one-shot generation (without conversational history) will still produce different results due to sampling randomness. More critically, during inference a model may be subject to server-side A/B routing — the same API endpoint may call different weight versions or system prompt versions at different times, completely opaque to the user. This makes "was it randomness or did the model change?" a question that is, in principle, indistinguishable from the user's side. To rule out randomness with statistical significance, you'd need to take a large number of repeated samples from the same model version within one time window and compare their distribution against samples from another time window — not rely on a single-sample comparison across two time points.

The "Quiet Model Swap" Accusation and Calls for Transparency

The author goes further, floating a somewhat conspiratorial theory: maybe users during the first three days were actually running on a stronger model (which he calls "Mythos") that was quietly swapped out afterward. He also mentions that compared to 4.5/4.6, Max effort mode "overtests" and takes too much control away from users.

Regardless of the truth, his core demand is transparency — users have a right to know exactly what model they're calling and whether any changes have been made.

It's worth emphasizing that "post-launch degradation" is a recurring collective experience in the AI community, but rigorous third-party verification has yet to materialize. Vendors typically explain performance fluctuations with routing strategies, safety adjustments, load balancing, and similar rationales, while users' subjective experiences are difficult to clearly distinguish from objective regressions. The author has promised to upload both versions' source code to his personal dev blog and share a GitHub link in the comments, leaving the door open for independent community verification.

How to Evaluate This Kind of "Nerf" Evidence

The value of this case isn't that it "proves" Opus 5.5 was degraded — a single sample clearly isn't enough for that. Its value lies in demonstrating a more responsible approach to raising concerns: controlling variables, preserving original artifacts, digging into the code for root causes, rather than stopping at emotional venting about how things "used to be better."

For developers who rely on AI coding tools, a few practical takeaways: important generated outputs are worth archiving; don't treat one-shot generated code as a black box you can blindly trust — the underlying engineering quality needs human review; and maintain a reasonable, verifiable skepticism toward vendor capability claims. Until vendors provide clear explanations, posts like this remain unverified single-source accusations, and readers should proceed with caution.