Claude Opus 5.5 Hands-On: From Sketches to 3D Worlds, AI Is Reshaping Game Development

Anthropic, even amid growing calls in the industry to slow down AI development, went ahead and launched Opus 5.5. Bilibili creator Pursuing AI recently published a research video showcasing a series of outputs from this model — from interactive physics simulations to fully explorable 3D worlds. These examples point to a clear trend: AI coding is shifting from generating scattered code snippets toward building increasingly complete, interactive experiences.

This article is based on that video, walking through Opus 5.5's most representative demos while examining the real capability limits and true costs behind them.

Interactive Lens Lab: Turning Abstract Concepts Into Playable Demos

The first case comes from user Rianne Sayer, who asked Opus 5.5 to build an interactive lens lab to explain camera focus. Rather than outputting a wall of text, the model constructed a visual, interactive simulation — a complete lens structure, individual optical elements, a focal plane, a background scene, and control buttons for adjustable parameters.

The most compelling part: when you move the focus point, the model visually shows how the plane of sharpness shifts through the scene. Instead of reading a dry explanation of depth of field, you watch the concept unfold in front of you.

The cost data is worth noting: according to the poster, generating this took 1 hour and 26 minutes with an API cost of $25.66. This isn't a simple webpage that pops up in seconds — it consumed significant time and compute. What's truly impressive isn't that it produced a nice-looking interface, but that the model translated the teaching goal of "explain focus" into a structured, interactive visualization.

This is not a perfect physics simulator, of course — it's a carefully crafted interactive demo tuned for a specific educational purpose. For that goal, it achieves something text simply cannot.

Browser Basketball Game: From Idea to Playable 3D Prototype

The second case is a browser-based basketball game. The creator asked Opus 5.5 for something like an NBA 2K experience, and the result included a 3D basketball court, a controllable character, a basketball, a scoreboard, a shot clock, shooting mechanics, movement controls, and an actual shot system.

The key distinction: this isn't a static model — in the video, the character runs around the court, dribbles, shoots, and interacts with the game. To be clear, this is not NBA 2K running in a browser. The post explicitly noted that the result was built using Three.js alongside GenX Games' agentic game development tools. So Opus 5.5's contribution should be understood as one piece of an entire development pipeline — it helped generate the game through code and agentic tooling rather than independently supplying all assets, systems, and infrastructure.

Compared to the real NBA 2K, this prototype has simpler visuals, more basic animations, and far more limited gameplay systems. But the point is: you say "give me a basketball game" and you get a working 3D browser prototype. That's a completely different experience from simply asking AI to generate a few lines of JavaScript.

Three.js is an open-source JavaScript 3D graphics library that lets developers render three-dimensional scenes directly in the browser via WebGL, with no plugins required. It abstracts low-level GPU calls and provides high-level APIs for cameras, lighting, materials, geometry, and animation — making it the go-to choice for browser-based 3D games and visualizations. GenX Games' agentic game development tools represent a new class of "AI agent frameworks": these frameworks decompose large language models into multiple self-planning, tool-calling, collaborating "sub-agents," each handling a subtask (generating maps, writing physics logic, debugging collision detection), with the outputs assembled into a complete project. This distinction matters — the demo doesn't show Opus 5.5 generating an entire game in one shot. Instead, it acts as the "brain" within an agent framework, continuously breaking down tasks and driving multiple rounds of code generation and iteration.

Explorable Tropical Island: A Complete Environment for Nearly $1,900

The demo that best illustrates the upper bound of scale is a tropical island environment shared by user Dan Green. He spent $1,874.40 in Opus 5.5 tokens — nearly $1,900 — to build a playable island scene.

According to his description, you can walk around the island, explore the beach, walk out to a dock, take a boat out, and even dive underwater. The environment is richly detailed: houses, vegetation, ocean, docks, boats, dynamic lighting, and a full day-night cycle. At night, lights illuminate the dock and surrounding areas; underwater, you can spot marine life including whales. Multiple systems are integrated into one interactive experience.

The author openly acknowledges bugs and visual imperfections — some textures need work, and the shoreline waves look a bit off in places. This is clearly not a commercially polished open-world game.

The production method is equally telling: the entire project used multiple sub-agents, ran for about 2 hours and 7 minutes of API time, and translated to roughly 8 hours of real-world time. The author said they gave almost no technical direction — most prompts were simple, like "add this" or "this doesn't look right, fix it." The AI handled the vast majority of building and iteration. But this also makes clear that a single sentence won't generate a AAA title — behind the scenes is real compute, multi-agent collaboration, and hours of iteration.

The "token cost" mentioned here is fundamental to understanding AI usage fees. Large language models bill by the "token" — the basic unit after text is tokenized. One English word corresponds to roughly 1–2 tokens; Chinese characters typically map to 1–2 tokens each. Anthropic's API prices input and output tokens separately. For complex tasks involving heavy code generation, multi-turn conversations, and multi-agent collaboration, the model must read the full context at each step (input tokens) and generate new content (output tokens) — costs compound rapidly with project scale. The tropical island case cost nearly $1,900 precisely because multiple sub-agents iterated continuously over roughly 8 hours, accumulating massive token consumption with every round. This means real-world costs for complex projects need serious evaluation from the very start of planning.

Sketch-to-Simulation: From Visual Concept to Structured Physics System

The sketch-to-simulation demo showcases a distinctly different capability. The creator drew a rough sketch showing a launcher, projectiles, and a pair of blocks — and Opus 5.5 turned it into an interactive 3D simulation.

This goes beyond converting a drawing into a 3D model. The launcher operates like a real physics system — you can adjust parameters, watch the projectile trajectory, fire it, and see the blocks topple on impact, with measurements like mass and velocity displayed throughout.

The real leap here: the model translated a visual concept into a structured environment, then layered behavior and simulation on top. That's far more complex than generating a static image from a sketch. Whether the physics calculations are fully realistic is hard to verify, but as a capability demonstration it makes its point clearly.

The "sketch-to-simulation" capability involves multimodal input: the model must not only "read" the spatial relationships and semantics of elements in a hand-drawn image (which is the launcher, which is the target), but also map that visual understanding to structured code representations — including scene graphs, object properties, and physics engine parameters. Modern physics simulations typically rely on JavaScript physics engines like Cannon.js, Ammo.js, or Rapier, which handle rigid body collisions, gravity, and velocity decay. Opus 5.5 acts as a "translation layer" here: parsing visual concepts into engine-readable parameter configurations, then wrapping them into a complete interactive scene. This is fundamentally different from pure text-based code generation — it requires the model to simultaneously coordinate image understanding, spatial reasoning, and physics modeling.

Spider-Man-Style Web-Swinging Game: A Complete 3D City Generated From Scratch

The most visually striking demo is a Three.js game built with Opus 5.5 Medium. The creator claimed all 3D models, textures, and animations were made from scratch — the result is a Spider-Man-style web-swinging game set in a large three-dimensional city.

You can sprint across rooftops, leap between buildings, swing through streets on web lines, and scale glass facades. The most impressive aspect is environmental consistency — this isn't a single scene. The city remains coherent throughout the entire playthrough, with buildings, roads, traffic, trees, pedestrians, streetlights, and a distant skyline, plus a minimap, mission objectives, and a real HUD.

The web-swinging mechanic is the standout: the character attaches to buildings, swings around corners building momentum, then releases to glide between skyscrapers. It's no Spider-Man 2, of course — animations are simpler, models are repetitive, physics aren't professional grade, and NPCs and environments show visual repetition. But for an AI-assisted development pipeline, turning "a game description in your head" into something you can actually play is already remarkably close to the goal.

The Direction of AI Coding: From Writing Code to Building Worlds

These cases represent only a portion of Opus 5.5's capabilities. From interactive simulations and browser games to complete 3D worlds, the most notable shift is this: we're no longer just asking AI to write code — we're asking it to turn an idea into something visible, usable, and interactive.

The limitations deserve a clear-eyed look: the demos generally take a long time, rely on multiple agents and supporting tools, carry token costs ranging from tens to nearly two thousand dollars, and the results contain noticeable bugs and visual imperfections. Don't expect to describe your next GTA and have a perfect game materialize in five minutes.

But the direction is clear — AI coding is shifting from generating scattered code snippets toward building increasingly complete interactive experiences. The core question is no longer "can AI write code?" but rather "can you describe what's in your head well enough for AI to turn it into something you can actually play?" The Opus 5.5 demos are steadily bringing that answer into focus.