Local AI Comic Drama Production with ComfyUI: Complete Workflow from Images to Final Cut

Introduction: The Reality and Ideals of AI Comic Drama Production

Many people have a misconception about AI comic drama production — they think a simple prompt is all it takes for AI to automatically generate a complete storyboard and finished video. But the reality is that neither open-source nor closed-source models can currently achieve this. Bilibili creator "Programmer Luobo" (程序员萝卜), in tutorial episode #268, used a Marvel hero interview short film as an example to break down the complete workflow for 100% local AI comic drama generation, requiring only a 4060Ti GPU.

The core value of this tutorial lies in the fact that it's not empty theory — it's a battle-tested, reproducible production pipeline. The entire workflow involves three core tools: the LTX 2.3 video model, the KLEIN image editing model, and the QWEN TTS voice model.

Image Preparation: The Foundation of AI Comic Drama Production

Why Images Are Key

In LTX first-last frame mode, all critical actions need to be pre-expressed through images. This means you can't rely purely on prompts to have the model automatically generate complex actions — movements like the Hulk picking up a girl or Iron Man fist-bumping with a girl all require pre-made corresponding images.

Technical Background: LTX First-Last Frame Mode LTX (Latent Text-to-video eXtension) is a class of diffusion model-based video generation architectures. First-Last Frame Conditioning refers to providing both the starting and ending frames as constraints during video generation, letting the model perform motion interpolation and content filling between the two frames. This approach essentially transforms the video generation problem into a "constrained temporal interpolation" problem. Compared to purely text-driven video generation, it drastically reduces the space where the model needs to "improvise," resulting in significantly improved character consistency and visual stability. The tradeoff is that creators must pre-design keyframe images, raising the production barrier from "writing prompts" to "mastering image editing."

Using the Marvel hero interview short film as an example, a 20-second interview segment for one hero requires 5 images. The production workflow is as follows:

1. Generate base images: First generate the girl's baseline images
2. Composite character images: Use the KLEIN model to generate photos of the girl with each hero
3. Design action images: Generate separate images for each key action (e.g., picking up, touching chest muscles, etc.)
4. Image correction: Use KLEIN's single-image editing to fix details (e.g., adding metal gauntlets to Iron Man)

KLEIN Model Image Editing: Practical Tips

The image editing uses the KLEIN single-image editing feature introduced in tutorial episode #230. KLEIN is an instruction-driven image editing model, an evolved version of the InstructPix2Pix paradigm. Its core capability is making local or global modifications to existing images based on natural language instructions, without requiring users to manually draw masks. Compared to traditional inpainting methods, KLEIN can understand semantic-level editing intent, such as "add metal gauntlets to the character" or "composite two characters into the same scene." The model's advantage is extremely fast speed — completing one edit in just 13 seconds — thanks to its lightweight inference architecture design, enabling rapid iteration even on consumer-grade GPUs. However, note that not every generated image is usable; you need to repeatedly "roll the dice" until you get a satisfactory result.

Here's an important mindset shift: To produce good AI video, you must master basic image editing skills. The idea of skipping image editing entirely and relying solely on video models to produce finished content is virtually impossible under current technical conditions.

LTX First-Last Frame Mode in Detail: Strengths and Limitations

Core Advantages

• Fast speed: Runs smoothly on a 4060Ti GPU
• Long video support: Can generate relatively long continuous videos through first-last frame chaining
• Good character consistency: Character consistency in first-last frame mode far exceeds that of pure image-to-video generation

Limitations Not to Be Ignored

• Micro-stuttering at junctions: There will be slight stuttering at the junction between each first-last frame pair — this is an inherent defect
• Poor prompt responsiveness: Since the first and last frames already determine most of the motion state, prompts have limited influence
• Limited camera movement: Camera movement effects need to be achieved through the first-last frame images, not through prompt control

How to Write Prompts Correctly

Prompts should be concise but precise. For example: "A woman smiles and talks, Marvel hero Deadpool walks into the frame from the right" — the words "walks into the frame" are crucial. Without them, the model might use abrupt cuts or transitions to place the character in the scene, which looks very unnatural.

Don't write overly complex prompts — even if you do, the model can't execute them. Those approaches that feed complex storyboard scripts into LTX first-last frame mode have an extremely low probability of actually producing usable results.

Voice Generation and Synthesis: Making Characters Speak with QWEN TTS

QWEN TTS Voice Generation Strategy

The voice component uses the QWEN TTS model from tutorial episode #257. QWEN TTS is Alibaba's Tongyi Qianwen series text-to-speech model, a neural network TTS system driven by large language models. Unlike traditional TTS, LLM-based TTS can understand context, emotion, and intonation, generating more natural speech. Voice characteristics (timbre, speed, emotional tendency) are controlled by descriptive text in the prompt.

Here's a key technique: All lines for the same character should be generated in one batch, rather than in separate segments.

The technical reason: within a single inference pass, the model maintains a relatively stable latent representation, ensuring timbre consistency. Multiple independent inferences may cause voice drift due to random seed differences and context variations. For example, all of the girl's lines (opening remarks, interview questions, closing statements) are written in one workflow and generated all at once, resulting in 36 seconds of complete audio.

The three Marvel heroes, being different characters, need to be generated separately using different workflows, with voice descriptions adjusted in the prompts (e.g., "middle-aged male voice, Western accent, teasing tone").