Testing GPT-6 Astra: Can It Reproduce the Bad Apple Animation in One Shot?

From Benchmark Hype to Hands-On Testing

GPT-6 Astra has sparked intense discussion across the AI community recently. OpenAI claims its capabilities approach several frontier metrics, and its benchmark scores are undeniably impressive. Demo videos and fan-made content have been flooding social media — but high scores on benchmarks don't automatically translate to strong real-world performance. The real test always lives in specific, concrete tasks.

For this experiment, I chose a particularly challenging target: asking GPT-6 Astra to generate an animation in a single pass that recreates the classic Bad Apple MV. With its pure silhouette style, high frame count, and continuous motion, Bad Apple has long been considered a gold-standard test for rendering quality and animation coherence.

Why Use Bad Apple as the Test?

What makes Bad Apple difficult is that it's a complete, dynamic visual sequence composed entirely of silhouettes — placing high demands on a generative model's temporal consistency and frame-to-frame stability. I personally recreated Bad Apple by hand during a puzzle/building event in a game not long ago, so I know firsthand just how much effort and fine-grained control this kind of task requires.

Astra is reportedly strong at "single-pass animation generation" — which is exactly the core claim this test was designed to verify: can the model maintain consistency across continuous frames in one generation, rather than simply stitching together a series of static images?

Bad Apple!! is a song-and-animation work born in 2009 from the Touhou Project doujin community. The original track was composed by Masayoshi Minoshima and performed by nomico, with animation by anira. It depicts a large cast of Touhou characters through smooth transitions and morphing shapes, rendered entirely in stark black-and-white silhouettes. Despite its minimalist visual style — literally just black and white — the motion is complex and the frame density is high. Its iconic status on the internet has made it a universal technical benchmark: from Minecraft redstone circuits to oscilloscopes, Game of Life cellular automata, and beyond, nearly every system capable of producing images has been used by someone to run Bad Apple. This cultural context means "can it run Bad Apple?" carries an inherent sense of pushing a system to its limits — making it both a community-validated and sufficiently challenging target for evaluating AI animation generation.

Using SVG to Avoid Simply Copying Existing Assets

To prevent the model from directly retrieving or reproducing footage from the original MV, this test specifically required it to draw the animation using SVG (Scalable Vector Graphics). This constraint serves a dual purpose:

• Verifying genuine generative ability: SVG rendering requires the model to understand the structure of each frame and describe it in code, rather than retrieving existing image assets.
• Evaluating code-based output: Vector drawing tests the model's ability to translate visual content into structured code, which more directly reveals its underlying level of comprehension.

This approach provides a fairer assessment of whether the model truly "knows how to draw" versus "knowing how to look things up."

SVG (Scalable Vector Graphics) is an XML-based open standard format that describes points, lines, curves, and shapes in code rather than storing pixel data. Unlike raster formats (PNG, JPG), SVG scales without any loss of quality at any resolution, and its contents are fully human-readable and auditable. For an AI model, generating SVG means translating visual intent into precise geometric coordinates and path instructions (e.g., <path d="M 0 0 L 100 100"/>). This demands genuine "understanding" of the scene's structure — not image retrieval or style transfer. Bad Apple is dominated by flowing, organic silhouette curves, making SVG path rendering far harder than drawing simple geometric shapes. Imposing the SVG format as a constraint effectively cuts off the shortcut of "memorizing and reproducing images seen during training," making it a particularly well-designed testing condition.

Thinking About Real Capability Behind High Benchmark Scores

AI models today are often marketed primarily through benchmark scores, but a gap consistently exists between those scores and actual output quality. An animation reconstruction task like Bad Apple exposes weaknesses in long-sequence consistency and detail fidelity far more effectively than straightforward Q&A or single-image generation.

Judging by community activity, fan creations and demos centered on Astra are already plentiful — but cases that apply strict constraints (such as prohibiting pre-existing assets and requiring SVG output) remain rare. This is precisely where hands-on testing adds value: it offers a reference dimension that benchmark numbers simply cannot provide.

AI benchmarks are typically designed to measure specific sub-capabilities — mathematical reasoning (MATH, GSM8K), code generation (HumanEval), knowledge Q&A (MMLU), and so on. These test sets can become over-optimized during training or alignment, leading to "score inflation" where a model's benchmark performance far outstrips its real-world utility. The industry calls this "benchmark contamination" or overfitting. Open-ended, long-sequence creative tasks like continuous animation generation have no standard answers to memorize, and they impose sustained demands on contextual consistency — naturally bypassing most score-gaming strategies. They serve as an effective complementary dimension for measuring a model's true generalization capability.

Conclusion

Whether GPT-6 Astra lives up to its reputation requires more real-world tasks like this one to find out. Testing it by reproducing Bad Apple in SVG offers a practical approach: sidestepping asset copying while focusing squarely on genuine generative ability. For anyone tracking AI generation capabilities, watching how a model handles a tough, constrained task tells you far more than any benchmark score.

(Note: This article is based on a hands-on test video by a Bilibili creator. For specific generation results and details, please refer to the original video demonstration.)