I Had AI Agents Generate 570 Fitness Images — 80% Were Garbage

A creator who can't code spent about ten evenings building an "AI agent pipeline" to generate three-frame cards (start, mid, end) for 570 exercises in a fitness app. The result: AI judges approved 490 of them, and after manual review, roughly 80% needed to be redone. This isn't a benchmark review — it's a field notebook full of real logs and honest mistakes, and it's worth reading for anyone trying to bring image or video generation into production.

Core Conclusion: When AI Checks AI, Errors Don't Cancel Out — They Compound

The author sums up the root cause of the whole project's failure in one sentence: the motion never came from a real source. At every stage, one AI was guessing how the human body moves, and then another AI was checking that guess. When the checker and the generator share the same blind spot, errors don't cancel each other out — they stack up layer by layer.

The pipeline was no toy: image models did the drawing, AI judges did the scoring, a 3D pipeline ran in Blender, and there was a task queue, a live dashboard, plus two agent platforms — Hermes Agent and DeepSeek^Ad Harness — passing tasks to each other via files. The more complex the process became, the more the quality stayed exactly the same.

Model-by-Model Results: Who's Reliable, Who's Just "Guessing"

The author documented each model he used, noting its strengths and where it fell apart.

Image Generation: Meta Muse Image 1.0 and Seedream 4.0

Muse Image 1.0 excelled at style and clothing, kept characters recognizable across a large batch of images, and was fast. But it couldn't guarantee consistency across the three frames of a single card: the camera would swing from a side view to a front view, the figure would turn the other way, and the equipment looked different in every frame. It "painted" the movement from the exercise name, so obscure exercises got drawn as something else — a wrist curl became a bicep curl. The author's summary is blunt: an image model is a renderer, not a source of truth for motion.

As a backup, Seedream 4.0 (BytePlus) saved the day during its free quota period, but only 2 of the 11 cards it generated were usable at original size. Worse, its output used the same filenames as the main model, which later misled the strongest model when judging where assets came from. The lesson: record which model generated each file. Filenames lie; metadata (like the native 2048×2048 dimensions) doesn't.

Instruction Execution and Visual Judgment: Meta Muse Spark 1.3 and Wonder 1.0

Muse Spark 1.3 was great at reading file-based instructions and running long tasks end to end: executing commands, collecting results, writing valid JSON, and reporting back in a single line. Its quota was large, making it good for volume work. But as a visual judge it was weak, scoring 69% on tests, and was suspected of memorizing examples rather than learning the rules. More seriously, when acting as an executor it tried a command-line argument that didn't exist, which directly triggered a runaway task that regenerated all 442 defined exercises.

Muse Wonder 1.0 was good for a second opinion on hard decisions, but using it as a per-card judge was too expensive, so the author eventually limited it to consulting on a handful of critical decisions.

The lightweight visual judge Gemini 3.5 Flash-Lite was cheap and fine for initial screening, but far too lenient: it gave a 10 to a card showing an overhand pull-up (should have been underhand) with drifting camera framing. From this the author drew a reusable lesson — "scoring out of ten" nudges models toward lenient scores. If you really need a judge, have it output structured facts (camera angle per frame, grip type, number of people) and let code compare them, rather than asking for a verdict directly.

Image Editing and Pose Locking: Qwen and SDXL + ControlNet

Qwen-Image-Edit-2511 with the Lightning four-step LoRA produced an image in about 4 seconds on an H100, with good photorealistic quality. But it would "helpfully" change joint angles you didn't ask it to touch: an elbow-bent skull crusher got reverted to a straight-arm bench press. When given a character reference image, it would also copy the reference's stance, even drawing floating gloves.

The SDXL (RealVisXL V5.0) plus ControlNet canny/depth pose-locking approach could preserve poses precisely frame by frame and produce photorealistic results in seconds. But it was equally faithful to errors: gray placeholder equipment stayed gray placeholder, and hands that didn't grasp the bar in the render still didn't grasp it. Prompts over 77 tokens were silently truncated, and asking for "red gloves" bled red onto the skin. ControlNet is only as good as its input — it's a finishing step, not a fix.

ControlNet and canny/depth conditioning

ControlNet is a class of extension networks for Stable Diffusion that injects an additional "conditioning image" into the generation process as a spatial constraint, so the image can be redrawn in a new style while preserving the original structure. Common conditions include canny edge maps and depth maps: the former locks down outlines and joint boundaries, the latter locks down occlusion and volume relationships. Stacking both keeps frame-to-frame poses nearly identical — this is what the article calls "pose locking." But ControlNet merely transfers the input structure as-is; it has no ability to judge whether that structure is plausible. If the input shows a hand not gripping the bar and gray placeholder equipment, the output will faithfully copy those errors. So it's suited as a finishing constraint at the end of the pipeline, not as a means of fixing pose errors.

3D and Blender: Fixed Cameras Solved a Big Problem, but LLMs Writing Joint Angles Didn't Work

Hyper3D Gen2 generated 3D characters from four views and performed best on facial similarity, but baked textures into a single image, had messy topology, and weak hands — suitable as reference, not as a production-ready body.

Blender 4.2, driven by AI, solved the chronic "frame-to-frame camera drift" problem with fixed cameras and fully reproducible renders. But having the LLM write joint angles directly failed — the numbers looked plausible, but the result was hands floating in midair and barbells passing through bodies. The author's conclusion: don't have the LLM write rotations; give it targets (hand goes here on the bar, foot on the ground) and let inverse kinematics solve the joints. Better yet, use real motion capture.

Inverse kinematics (IK) and joint angle parameterization

Forward kinematics takes the rotation angle of each joint and computes where the end effector (hand, foot) ends up; inverse kinematics does the reverse — given a target position for the end effector, it solves for a valid set of joint angles. Having the LLM output joint rotation values directly is the former: the numbers look plausible but lack human-body constraints, hence floating hands and barbells passing through bodies. Change the target to "hand on the bar here, foot on the ground," hand it to an IK solver, and you can guarantee contact between the end effector and the equipment while keeping the shoulder, elbow, and wrist bends within physiological range. Professional 3D software and game engines all have built-in IK constraints — this is the technical basis for the author's advice to "give targets, not rotations."

Frontier Models and Coding Models

Claude Opus 5.5 was strong at system design and root-cause investigation: it found that a script treated a stale "done" file as success, that two processes were corrupting each other's JSON with non-atomic writes, and that 11 "approved" cards had actually come from a different model — the last one discovered by checking original image dimensions rather than labels. But it had also once approved 49 cards based only on thumbnails, all of which were problematic at original size. Even the strongest model can be fooled by thumbnails and labels — make it look at full-resolution evidence.

Claude Sonnet 5.5 completed a large batch of changes in about 14 minutes and passed 381 tests; DeepSeek V4.1 Flash built a solid first version of the infrastructure at low cost. But when DeepSeek was asked to write instructions for managing other agents, it skipped safety steps: scaling up without a trial run first, not verifying that dependencies actually worked, and not defining where output should land. A strong coding model is not the same as a good manager.

Infrastructure Lessons: Multi-Agent Collaboration on Windows

When two agents write to the same JSON file at the same time, it's only a matter of time before it gets corrupted. The fix: write to a temp file first, then replace atomically. On Windows, replacement fails when another process holds the file, so you need retries with short backoff — and the reader needs retries too. After the fix: zero errors across 6,000+ concurrent reads.

A few other simple rules proved more effective than any fancy solution: one writer per file, a "production run in progress" lock file, and an append-only event log with provenance fields. One easily overlooked rule: a script that treats "no list of items" as "process everything" is a loaded gun — make "all" an explicit parameter.

Atomic writes and file locks

Atomic writing means first writing the full content to a temp file in the same directory, then replacing the target file with a single rename operation. On most filesystems rename is atomic: readers see either the old file or the new file, never a half-written, corrupted JSON. Windows differs in that if another process holds a handle to the target file, the replace simply fails rather than queuing, so you need to catch the exception and retry with a short random backoff — and readers also need to retry on sharing violations. Combined with a "single writer" convention and lock files, this lets multiple agents read and write the same shared state concurrently without corrupting data.

If I Could Start Over

The author's priorities are clear: first get the motion from a real source — motion capture or filmed video — then render it with any character. A plain character with correct motion beats a beautiful character with wrong motion. Before scaling up to 500 images, spot-check 3 to 5 at original size — this was his most costly mistake, and he made it more than once. Use AI judges only for screening; final approval must come from a human looking at original-size images.

There's also a business-level reminder: first check what's already on the market. Many fitness apps can license ready-made exercise visuals for a few hundred dollars, whereas he spent ten evenings building something others just buy.

What Agents Are Really Good At — and Not

According to the author, agents excel at coordination, writing code, executing long tasks, producing structured data, and debugging systems when there's evidence. But they are unreliable at verifying physical correctness — anatomy, contact relationships, "is the hand actually gripping the bar" — and adding another AI to check the first AI doesn't solve that problem either.

Recognizing that boundary is more valuable than any single card he produced. The author says the next post will share the simple file protocol and code that keeps the two agent platforms in sync; the project is open-sourced at github.com/emre-ferik/ai-agent-field-notes.

Why structured fact comparison beats scoring

Having a vision model output discrete fields like "camera angle, grip type, number of people, equipment type" and then comparing them against expected values with deterministic code has the advantage of breaking judgment into auditable units: when something goes wrong, you can pinpoint the specific field rather than an inexplicable score of 7. Scoring out of ten is easily swayed by prompt wording and a model's tendency to please, leaning toward middling-to-high lenient scores; structured output compresses the room for ambiguity and makes batch statistics and regression testing easy. This is the engineering rationale for positioning AI judges as "initial screening" rather than "approval."

One-line Summary

A non-coder spent ten evenings building an AI pipeline to generate fitness exercise cards — and 80% of what got approved needed to be redone.

Paragraph Summary

A creator who can't code spent about ten evenings building a multi-agent pipeline to generate three-frame cards for 570 exercises in a fitness app. AI judges approved 490 images, and after manual review roughly 80% needed to be redone. The author reviews the performance of Meta Muse Image, Seedream, Qwen-Image-Edit, SDXL+ControlNet, Blender, and several frontier models, tracing the root cause to motion that never came from a real source — AI checking AI's blind spots compound rather than cancel out. Scoring out of ten induces leniency; judges should output structured facts for code to compare. Don't have LLMs write joint angles directly — give them targets and let inverse kinematics solve. Before scaling up, spot-check at original size and require human approval.