Hands-On with GPT-5.6: A Single Prompt Builds a Real-Time Voice Anime App

OpenAI GPT-5.6 In-Depth Review: A New Yardstick for the Agentic Era

OpenAI's newly released GPT-5.6 has been described by this Bilibili creator as a generation of models that is "almost ridiculously powerful." This time it comes in three tiers: the most powerful and most expensive Sol, the mid-range Terra, and the smallest and fastest Luna. To test its true limits, the creator didn't bother with routine tasks like writing emails or summarizing text—instead, they dropped it straight into the Codex agent framework and put its boundaries to the test with a series of high-difficulty, real-world challenges.

A Model Designed for Agentic Scenarios

While GPT-5.6 can be used directly in the ChatGPT web version, unleashing its full capabilities—especially when it comes to collaboration across multiple files, tools, and steps—is better done by running it inside a harness like Codex.

What is an Agentic Framework? An agentic framework refers to a system architecture that runs a large language model not merely as a Q&A tool, but as an autonomous "agent" that executes multi-step tasks. Within such a framework, the model can iteratively call external tools (such as code executors, web search, file systems, and API interfaces), dynamically adjusting its next action based on intermediate results until it accomplishes the final goal—forming a complete "perceive-decide-act" closed loop rather than a single input-output exchange. The core idea behind this architecture originates from the "agent-environment interaction" paradigm in reinforcement learning: at each step, the model perceives the current state (such as code error messages or tool return values), makes a decision (which tool to call, what code to write), executes it, then observes the new state and continues. Compared with the traditional "one question, one answer" mode, agentic frameworks upgrade the LLM's role from "information extractor" to "task executor"—this is what the industry commonly refers to as the paradigm shift from LLM to Agent. Beyond Codex, mainstream agentic frameworks today also include LangGraph, AutoGen, and CrewAI, each with different emphases on tool-calling protocols, multi-agent collaboration, and fault-tolerance mechanisms.

The Codex app has now been renamed the ChatGPT Desktop version and is available as a free download across all platforms. Its biggest feature is support for multiple agents processing different folders simultaneously, along with the ability to save and reuse functions and set up scheduled automation tasks. This design fits perfectly with GPT-5.6's core positioning—it was built for agentic scenarios from the ground up, capable of autonomously calling various tools and running an entire workflow from start to finish.

Generating a Real-Time Voice Anime Character with a Single Prompt

The first test was quite challenging: build a role-playing web app featuring an anime girl capable of real-time voice interaction—her avatar generated by GPT Image with different mouth shapes stitched into a talking animation, while integrating Google Gemini's real-time voice model to achieve Lip Sync.

The technical principle behind Lip Sync: Lip Sync is the technique of aligning audio signals with corresponding mouth-shape animation frames. This technology dates back to the frame-by-frame hand-drawn practices of Disney animation in the 1930s, later evolving with the development of computer graphics into automatic alignment algorithms based on phoneme classification—parsing speech signals into basic pronunciation units like /m/, /a/, and /s/, then mapping them onto a predefined set of mouth shapes (visemes). High-end solutions in the current deep learning era (such as SadTalker and Wav2Lip) can generate highly realistic lip-motion videos directly from any face image and audio, but at the cost of heavy computation and high latency. The approach mentioned in this article is much lighter: pre-generate several static mouth-shape images with GPT Image (closed, slightly open, wide open, etc.), then dynamically switch between corresponding frames based on the phonemes or energy features output by Gemini's real-time voice model, achieving a low-cost pseudo-dynamic effect. This "state machine switching" approach sacrifices precision in exchange for high responsiveness to real-time voice streams, making it a classic engineering trade-off for resource-constrained scenarios.

The creator's workflow was quite streamlined: in Google AI Studio, select the Realtime model and voice, copy the code as a reference, fill in the API Key, choose the largest 5.6 Sol model, and crank the thinking intensity up to Ultra.

In the end, the model ran for 17 minutes, automatically using GPT Image to generate avatars with different mouth shapes—simply open the URL to start chatting. In the demo, the anime girl smoothly told a corny joke, all accomplished with just a single prompt, without the creator asking follow-up questions or manually fixing any bugs. This "one-shot completion" capability is precisely what sets GPT-5.6 apart from previous generations.

Writing a Physics Engine from Scratch

The second test went even further: build a liquid splash simulation with adjustable gravity and lighting, plus hand tracking via webcam and gesture-based motion control. The key constraint was that it could not use Three.js, nor call any external library with pre-rendered physics animations—meaning the model had to write the entire physics logic from scratch at the lowest level.

What does writing a physics engine from scratch mean? Fluid simulation is a recognized hard problem in computational physics, with its mathematical foundation being the Navier-Stokes equations describing the motion of viscous incompressible fluids. In WebGL/Canvas environments, there are two common approaches to real-time fluid simulation: one is based on the Lattice Boltzmann Method (LBM), which discretizes the fluid into virtual particle groups that collide and propagate on a regular grid, well-suited for GPU parallelization; the other is the Stable Fluids algorithm proposed by Jos Stam in 1999, which ensures numerical stability through operator splitting and semi-Lagrangian advection, making it the mainstream solution in gaming and interactive media. Disabling high-level libraries like Three.js means the model must directly operate the low-level WebGL context, hand-writing vertex shaders and fragment shaders—which run on the GPU and handle geometric transformations and pixel shading respectively—while also implementing framebuffers and ping-pong texturing to store the flow field state, along with iterative updates of physical parameters like viscosity diffusion, pressure projection, and vorticity enhancement. This task extremely tests one's understanding of low-level mathematics and graphics, and being able to get it running correctly in a single prompt is almost unbelievably difficult.

The result: the model ran for 12 minutes with zero errors throughout, and the project worked immediately upon opening. The liquid effect was realistic, complete with parameters for flow energy, brush radius, gravity vector, viscosity, vorticity, lighting direction, and bloom, while the webcam hand tracking also worked normally. Tasks like these, which require low-level physics understanding, precisely demonstrate the advantage of Sol combined with Ultra thinking: often a single prompt gets it right the first time.

Autonomously Building an Unfamiliar Toolchain

Next, the creator tested its ability to use unfamiliar open-source tools. The task was to create a 16:9, roughly one-minute promotional video based on ByteDance's Cdream 5.0 Pro product page, with narration using Gemini TTS and the animation portion powered by an open-source tool called HyperFrames—without telling it how to install anything, providing only the GitHub link, and letting it figure out the setup by reading the documentation itself.

Thirty minutes later, the video was generated. Although there were issues such as text and images overlapping, poor layout, and occasionally garbled subtitles—and even after re-rendering with an additional prompt it still looked a bit rough—the fact that the model could autonomously read documentation, set up the environment, and run the complete pipeline was quite impressive. Similarly, it handled tasks like a DAW music interface (two prompts, generating a coherent 32-bar track with a Riser and a Drop), rendering complex 3D scenes, and using Manim to create Fourier epicycles drawing a butterfly (installing and running everything automatically in 19 minutes). Overall, its performance outperformed other frontier models—not professional grade, but with a remarkably high level of completion.

Medical Imaging and Visual Recognition: Failures and Surprises Coexist

Not all tests went smoothly. In a cancer imaging recognition test, faced with 6 scans containing tumors, GPT-5.6 misjudged nearly all of them as hemorrhages or incorrect types—essentially a complete failure. It also failed the classic "find the frog" test—currently, the only model that consistently passes this test remains the Claude series. However, the creator also noted: GPT-5.6 was at least willing to attempt answering biology-related questions, whereas Claude often outright refuses to answer when it encounters biology or cybersecurity topics.

On deep research tasks, GPT-5.6 performed excellently. When asked to write a technical report on the molecular driver mechanisms of a certain leukemia, the content produced after 31 minutes came with detailed tables, flowcharts, and cited references, covering core chapters like driver mechanisms, the evolution of targeted therapies, resistance mechanisms, and long-term follow-up—accurate and clearly structured. The newly added Work tab is equally practical, allowing connections to platforms like Slack, GitHub, and Google Drive to directly process files. For a task involving scraping the earnings reports of three tech companies and generating a comparison slideshow, it produced a professional and insightful result in just 26 minutes.

Benchmarks: Cost-Effectiveness Is Its Biggest Weapon

Based on the specs and benchmark data, GPT-5.6 focuses on agentic coding, complex reasoning, and hours-long autonomous task execution. The results across several benchmarks are as follows:

• Agent's Last Exam: Sol achieved the highest score, and is much cheaper than Claude Opus and Claude Fable;
• DeepSWE Long-Horizon Software Engineering Leaderboard: Sol Max ranked first, surpassing Claude Fable 5 with lower cost and token consumption (confidence intervals overlap, so statistical significance remains to be confirmed);
• ARC-AGI-2: Sol scored a high 92.5%, demonstrating an emergent ability to "learn new rules on the fly"; on the harder ARC-AGI-3 it reached close to 8%, while both Opus and Gemini scored below 1%;
• Hallucination rate: This is a clear weakness. On the Omniscience benchmark, 89% of questions triggered hallucinations, far higher than Claude Fable's 55% and open-source GLM5.2's 28%.

Why are ARC-AGI scores especially important? ARC (Abstraction and Reasoning Corpus) was proposed in 2019 by François Chollet, the creator of Keras and an AI researcher. Its design philosophy is rooted in his unique understanding of "what intelligence is"—true intelligence is not memorizing vast amounts of facts, but rapidly inferring new rules from minimal prior knowledge and generalizing them to unseen scenarios. Each ARC task provides only 3-5 input-output grid pairs as examples, requiring the model to infer the transformation rule and apply it to new inputs—rules that never appeared in the training data. This makes "drilling on practice problems" during pre-training ineffective; the model must perform genuine "on-the-spot reasoning." ARC-AGI-2 and the harder ARC-AGI-3 further increase rule complexity and the number of required reasoning steps, and are regarded by academics as a stress test "specifically designed to counter memorization-based generalization." Sol's high score of 92.5% on ARC-AGI-2—while the human average is around 85%—means that to some extent it demonstrated inductive reasoning abilities beyond pattern matching. However, academics remain divided on this conclusion, with some researchers arguing that sufficiently large-scale pre-training may still implicitly cover similar patterns.

The meaning of hallucination rate and the Omniscience benchmark: Large model "hallucination" refers to a model generating information that doesn't actually exist or is incorrect, with high confidence—it is one of the most serious reliability flaws in current LLMs. Mechanistically, the root cause of hallucination lies in the fact that language models are trained to maximize sequence prediction probability rather than factual accuracy—the model "naturally tends to generate plausible-sounding content" rather than "only saying what it's certain it knows." The Omniscience benchmark specifically measures a model's self-calibration ability at the boundaries of its knowledge by constructing a large number of factual questions that "the model shouldn't know" (such as fictional academic citations or undisclosed event details)—an ideal model should say "I don't know" when uncertain, rather than fabricate an answer. Mainstream strategies for mitigating hallucinations currently include Retrieval-Augmented Generation (RAG), Chain-of-Thought (CoT) guided self-checking, and adding reward signals for refusing to answer in Reinforcement Learning from Human Feedback (RLHF). GPT-5.6's trigger rate as high as 89% means it tends to "answer wrong rather than stay silent," which is a serious hazard in high-risk scenarios like healthcare and law, and fundamentally explains why its medical imaging judgments failed.

Notably, OpenAI has already used GPT-5.6 within its internal R&D process to accelerate its own research, forming a recursive loop of "using current-generation AI to help develop the next generation." This "Recursive Self-Improvement" mode is becoming a common practice among top AI labs: current-generation models generate code, experimental designs, and paper drafts, helping researchers validate hypotheses faster and thereby shortening the R&D cycle for the next generation. From a technical standpoint, this mode is still constrained by the fundamental limitation that "models cannot truly understand their own weight structures"—AI assists the efficiency of researchers' work rather than directly modifying the model itself. But as automated machine learning (AutoML) and neural architecture search (NAS) technologies mature, the depth of AI's involvement in designing its own architecture is gradually increasing. However, when models begin to deeply participate in designing their own successors, the effectiveness of human oversight also faces new challenges: can researchers still fully review the tens of thousands of lines of experimental code generated by AI? There is currently no consensus answer to this question.

Conclusion: A New Goal-Driven Agentic Paradigm

All things considered, GPT-5.6 is one of the most capable models currently available—cheaper and more efficient than Claude Fable 5, with speed that holds its own (although Sol's Ultra mode routinely takes anywhere from ten-plus to thirty minutes per task). Its most fundamental shift is this: it no longer needs to be hand-held—given a goal, it can autonomously make progress and work for hours until completion.

The weaknesses are equally clear: medical imaging judgment and fine-grained visual localization remain unreliable, the hallucination rate is high, and the generated videos, music, and 3D scenes still fall short of professional standards. But as a general-purpose model built for the agentic era, GPT-5.6's overall performance in coding, reasoning, long-task autonomy, and cost-effectiveness has already drawn a new upper bound on capability.

Key Takeaways