OpenReel: Open-Source Video Creation Studio That Turns One Sentence Into a Short Film, with ComfyUI Handed Off to an Agent

From a Single Prompt to a Finished Film: An Experiment in Agent-Driven ComfyUI

Bilibili creator Ayan (Ouyang) recently released an open-source video creation studio called OpenReel. The core idea is to hand off the complex scheduling of ComfyUI workflows to an AI Agent, letting creators focus purely on content. Built on top of the DeepSeek-Harness (DSH) framework, the tool is currently in beta and has already wired together image generation, audio generation, and storyboard-level (slideshow-style) production for narration-type videos.

Ayan openly acknowledges this is "essentially a beta" integration that hasn't yet unlocked ComfyUI's full potential — but it already demonstrates a complete end-to-end pipeline from a single prompt: creative brief, script writing, storyboarding, voiceover generation, image generation, and final video compositing. Once storyboards are in place, converting them to video "is already pretty straightforward," and downstream pipeline development is expected to accelerate.

Two Plugins, Two Roles: One Manages ComfyUI, One Manages Creation

Ayan places particular emphasis on the layered architecture. The entire system consists of two plugins, each with a distinct responsibility:

• The ComfyUI Plugin (previously developed by Ouyang): Manages the underlying workflows — the "infrastructure" layer. A small update was made for this release, adding a workflow preset import/export feature that lets users export a configured workflow along with its embedded Skills (which tell the Agent how to use the workflow and how to write prompts) as a file, making migration and integration much easier.
• The OpenReel Studio: Handles only "what content to generate and how to adjust it" — the upper creative layer.

The two exist as entirely separate dimensions. Ayanʼs philosophy is that ComfyUI serves as the underlying infrastructure where workflows can be freely customized, while the studio focuses on enabling genuine creative expression — "freeing us from the constraints and shackles of ComfyUI."

DeepSeek-Harness (DSH) is an Agent orchestration framework with a large language model at its core. Its design philosophy is similar to LangChain or AutoGen, but places greater emphasis on modularizing model capabilities into "Skills" that a scheduling layer can compose and invoke on demand. The framework is not tied to any specific model, though it is well-optimized for the DeepSeek model family. By building OpenReel on top of DSH, all instructions to ComfyUI pass through a "model → Skill routing → API call" chain rather than the user directly dragging and connecting nodes — which is both the source of its flexibility and the reason it has some dependency on network latency and model comprehension.

Version Compatibility: Use DSH 1.2 or Higher to Avoid Breaking Changes

One important technical detail is version compatibility. Ayan notes that DSH is currently at v1.25, and that around v1.2 the team introduced a "breaking change" that altered the protocol API interface. Plugins built before v1.2 cannot run on newer versions of DSH without having the language model adapt the protocol — or simply upgrading to v1.2 or above. Ayan recommends users "don't go below 1.2" to avoid plugin conflicts.

Audio Management Is Critical: Why TTS Audio Suite

Among all the ComfyUI compatibility requirements, image and video models are largely interchangeable — audio is the key constraint, because it involves a voice library management mechanism. Ayan chose the comprehensive audio plugin TTS Audio Suite from GitHub for its broad model support: F5-TTS, Chatterbox, Qwen (Tongyi Qianwen), Fish, Index, RVC voice conversion, and even sound effect blending.

You don't need to download everything — just install what you use. For this demo, Ayan used Qwen's TTS, specifically for its voice design functionality, which lets the Agent design a broadcast voice suited to the program's tone. The plugin comes with its own voice library, and the studio references this library for voice selection rather than hunting through scattered folders across ComfyUI. If you only need reference audio without a voice library, Index or F5 works fine and this plugin isn't required.

The TTS models integrated in TTS Audio Suite each have different strengths: F5-TTS and Index-TTS excel at natural Chinese prosody and support reference audio cloning (a few seconds of reference audio is enough to clone a voice); Chatterbox is Resemble AI's open-source emotional TTS, strong in English; the Qwen (CosyVoice) series offers multilingual support + a preset voice library + voice design capabilities, ideal for fine-grained stylistic control over voice tone; RVC (Retrieval-based Voice Conversion) is strictly speaking a voice conversion tool rather than TTS, typically used to convert existing speech into a target character's voice. OpenReel's core reason for choosing Qwen is its preset voice library and "voice design" parameters (adjustable age feel, emotional tone, speaking rate), enabling the Agent to automatically select an appropriate broadcast style based on program tone — rather than requiring the user to manually pick one each time.

Full Pipeline Demo: From The Crowd to a Finished Film

Ayan used a fairly substantive topic for the full demo — "the rise of high-cognition individuals and the dissolution of the collective in today's world" — drawing on social psychology concepts from The Crowd by Gustave Le Bon. The workflow unfolds in clearly defined stages:

Brief

After a single-sentence prompt, the Agent first produces a project brief: estimated length, overall structure (hook, segment breakdown), and style suggestions. Ayan didn't specify a length, letting the Agent "infer from the script content." The resulting project appears in the studio, where you can specify the target platform (horizontal/vertical, e.g., Xiaohongshu 3:4) and visual style (tech aesthetic, warm documentary, flat design, etc.).

Script and Storyboards

The next step generates a script, which is automatically divided into segments and storyboard shots. The demo produced around 25 storyboard shots, each labeled with tone (declarative statement, hook reversal, etc.) and an estimated duration. Ayan emphasizes that segments and shots are two different concepts — a single segment can be split into two shots corresponding to two separate images. The system also includes a standards-based review mechanism that calculates and evaluates timing, since creators "often have very little intuitive sense of time."

Voiceover Generation

For voiceover, users can either select a voice from the plugin library (e.g., Ayan's preset "30-year-old female voice") or let the Agent automatically design a voice suited to the theme. Generation supports single-shot or batch mode (all 25 shots queued at once), though a concurrency cap (around 10) means they're processed in batches. Ayan also added a trim function for clipping overly long silences at the beginning or end of audio clips.

Image Generation and Cinematography

In the image generation stage, after selecting a model, the Agent generates cinematography parameters for each storyboard shot: shot type, focal length, lens range, depth of field, lighting, color temperature (warm/cool), movement, and more — then merges these with the scene description into a final prompt. The workflow Ayan used natively generates at 1920×1080; quality is fine but generation is slow — "about two minutes per shot," meaning roughly 20 minutes total for 20+ shots.

Final Edit and Refinement

After storyboards are approved, they're imported into the editing interface for preview and trimming. Here you can adjust "pause" intervals within clips to control pacing, edit subtitles (extend/split/merge/shorten), add background music (Ayan connected a Music workflow, though admits the music output "still needs work"), and configure audio ducking (background music automatically lowers when narration is playing). The final output is composited via FFmpeg into a subtitled video.

Skill Three-Layer Architecture and the Human-AI Collaboration Philosophy

Ayan reveals that the narration pipeline alone contains around 11–12 Skills, organized into three layers:

• Tool-call Skills: Creating projects, placing assets on the timeline, dispatching content;
• Audit Skills: Evaluating quality of scripts, cinematography, pacing curves, etc.;
• Creative Skills: Responsible for actual creation — writing scripts, crafting cinematography language, etc.

One interesting detail: during the demo, the Agent invoked the wrong workflow twice — due to missing context in the conversation. But thanks to the Skill's error-logging capability, the Agent automatically records the mistake (e.g., a TTS parameter issue) and avoids repeating it next time.

Ayan repeatedly emphasizes his core philosophy: machines handle automated generation of outlines and content, while humans use a "workspace that's genuinely built for human creativity" to adjust and refine the details. He frames the whole project as an experiment in "how to leverage the open-source ecosystem" — rather than obsessing over 1080p vs. 4K or upscaling passes, the real bottleneck is "how we should create." To viewers, the difference between 1080p and 720p is minimal. "All the tweaking is in service of content — if there's no content, what's the point of higher resolution?"

In the DSH framework, Skills play the role equivalent to "Tools" or "Actions" in other Agent systems — but they additionally carry contextual descriptions that tell the model: what this tool does, when to invoke it, and how to fill in its parameters. The three-layer architecture (tool call → audit → creative) is essentially a separation-of-concerns principle: the tool layer ensures operations are executable, the audit layer ensures output quality meets standards, and the creative layer ensures content is meaningful. When the Agent invokes the wrong workflow and self-corrects, it relies on the audit Skill writing the error into the current conversation's context memory, allowing the model to reference past failures in its next decision — a lightweight "Reflection" mechanism that improves task success rates without requiring model retraining.

Roadmap: Video Narration, Multilingual Support, and More Pipelines

The current version is limited to illustrated narration (static image storyboards). Ayan's planned next steps include:

• Integrating MiniMax (Hailuo/H3) to generate actual video clips for a video narration mode;
• Multilingual support, such as generating English-language content;
• Using Web Research / Deep Research capabilities so the Agent can first gather relevant social discourse around a topic before writing more comprehensive, timely content;
• Pipeline categories already tagged in the GitHub repo include video ads, creative ads, short films, digital humans, episodic drama, and animation — all pending development.

A practical note on generation time: if using H3 for video, a typical consumer GPU averages "about one minute of generation per second of footage" — a 180-second film could take two hours to render. "Enough time to sleep through."

Ayan named the project OpenReel to signal that this is an open-source film and video creation studio — the goal isn't to teach people how to use ComfyUI, but to harness ComfyUI's open-source ecosystem in service of creative ideas. The beta is now available in the repository, with some known minor bugs (e.g., selecting a 3:4 platform but generating non-3:4 content causing aspect ratio issues). Ayan welcomes interested users to try it out and share feedback.

MiniMax's Hailuo/H3 is currently one of the higher-quality domestic video generation models. The H3 version supports continuous clip generation of up to tens of seconds, with notable improvements in motion consistency and image detail. Ayan's estimate of "one minute of generation per second of footage" corresponds to typical local inference speeds on a consumer GPU (e.g., RTX 4090); cloud API calls are faster but more expensive. Integrating H3 into the OpenReel pipeline would theoretically upgrade the current "static images + voiceover" narration format to "real video clips + voiceover" — but this introduces new challenges around cross-shot style consistency and mapping Agent prompts precisely to the video model's motion parameters, which will be the primary technical hurdles in subsequent development.