Codex Builds a JianYing Video Pipeline Automatically: A Zero-Basics Fully Automated Editing Walkthrough

Spending hours every day hunting for footage and aligning timelines is a shared pain point for content creators. A technical video on Bilibili demonstrates a "dimensional strike" approach: using Codex together with Python to generate JSON draft files that JianYing (CapCut's Chinese edition) can read directly in the background, letting the editing software run itself. The entire pipeline—from script, voiceover, and subtitles to timeline assembly—is fully automated, and the operator doesn't even need to open JianYing itself.

Core Architecture: Giving Editing Software an External Brain

The key cognitive shift in this approach is this: AI editing isn't about installing a "one-click generate" plugin into the software, but about bypassing the interface and writing directly into the project file at the底层 (底层/underlying layer). Codex acts as the central processor, using a high-level wrapper API to precisely deliver video tracks, audio tracks, effects, and subtitle tracks onto the timeline—essentially an automated distribution hub.

According to the Bilibili creator's explanation in the video, this approach replaces the clumsy method of simulating mouse clicks and drags—which breaks the moment you change resolution or a popup appears. Writing directly to the underlying API structure achieves millisecond-level positioning accuracy.

The automation engine's capability matrix covers six major areas: draft creation and overwriting, track-based import of video/images/audio, TTS voiceover and subtitle generation, title/transition/filter and keyframe injection, and invoking the toolchain for automatic screen recording and screenshots. Rather than a tool, it's more like a production line on standby.

The essence of JianYing draft files: JianYing serializes the editing state of each project (tracks, asset references, effect parameters, keyframes, etc.) into a JSON structure file stored in the local draft directory. It is decoupled from the UI operations—the UI is merely a visual editor for this data, reading and writing the same file. So as long as you generate valid JSON following JianYing's schema and place it in the correct directory, JianYing will load it as a normal project on startup. This path bypasses UI-layer event simulation, is immune to resolution changes, popups, and control-position shifts, and also makes batch generation and template reuse possible. The cost is that you must keep up with JianYing's version changes to the field structure, otherwise drafts may fail to load or some tracks may be lost.

Environment Deployment: Configuration in Plain Language

The most beginner-friendly aspect of deployment is that you don't need to write code by hand. Just give Codex natural language instructions, such as "help me install the GitHub Skill and configure it as a global skill, then create a Python virtual environment and install the dependencies," and the AI will automatically handle code download, dependency installation, and draft directory mounting.

Network issues are the most common stumbling block. The video offers a three-tier strategy: first, replace the installation source with a domestic mirror; second, enable a global proxy and retry; third, directly download the archive and guide the AI to deploy locally. Switch strategies as needed and restart the Codex skill to recover.

Minimum Viable Test and the "Ghost Draft" Phenomenon

Before going into full production, you must first do an MVP test: create a ten-second test draft, add a test video, BGM, and title, and check whether the three core tracks—text, video, and audio—work and whether any errors occur. Small steps and quick iterations are the only way to avoid large-scale rework.

After the test passes, you open JianYing and the draft list is empty—this is not a bug. The terminal shows the check passed, meaning the JSON draft is already sitting on the disk; it's just that JianYing's draft list doesn't monitor local file changes. The fix is extremely simple: close the software and restart, or click into an old draft and exit, forcing the software to re-read local data, and the "ghost draft" immediately appears.

Why JianYing doesn't monitor local file changes: Most desktop editing software scans the draft directory at startup and builds an in-memory project index; during runtime it does not use a file system watcher (such as inotify or FileSystemWatcher) for continuous synchronization. This reduces frequent IO and also prevents inconsistencies between the index and disk state caused by manual file changes. So a JSON newly written by an external process is completely invisible to an already-running JianYing; you must trigger a re-scan by restarting, or force an index refresh by opening and exiting another draft. Once you understand this, you can predict: any automation workflow of "background generation + foreground opening" must leave room for one index refresh action between writing and opening.

Full Production: The More Specific the Instruction, the More Perfect the Output

The core trick of the pipeline is the granularity of instructions. You can't just say "make me a video"; you have to break it down: first write the script and storyboard, specify the voice and BGM, prepare the corresponding screenshots and screen recordings, then add subtitles, transitions, and sound cues. The more demanding the conditions, the more the AI's background performance resembles a complete editing team.

The entire production line connects in four stations: script and storyboard structure first, TTS engine synthesizes the voice, tools like FFmpeg prepare the visual assets, and finally video, voiceover, BGM, subtitles, and sound effects are written into the timeline in one go. This is a leap from single-text generation to multi-modal engineering-grade assembly.

Why Not Remotion or Hyperframes

The video raises a key comparison: Remotion is based on React control, Hyperframes on the web, but both output a rendered final video—if there are typos in subtitles or you want to change a transition, it's nearly impossible to modify. The Codex approach produces an editable draft that retains all tracks; for self-media voiceover tutorials that require secondary creation, underlying write logic is currently the only correct answer.

Remotion and the "render-to-final-video" route: Remotion describes frames with React components and outputs MP4 through frame-by-frame rendering; tools like Hyperframes do compositing based on a web tech stack. Their commonality is that the final product is a pixelated video file; timeline information is flattened during rendering, and semantic information such as subtitle text, transition types, and keyframes is no longer retained as editable objects. Modifications require going back to the source code and re-rendering, and detail adjustments like subtitle typos or tone/pacing cannot be done in place in the editing software. The direct-project-file route defers "compositing" into the editing software, delivering a project with full track semantics, with nearly zero secondary editing cost—exactly the capability that voiceover tutorial content values.

The Boundary of Human-AI Collaboration and the Content Flywheel

The goal of this approach is not to replace editors, but to follow the 80/20 rule: AI takes on repetitive labor like aligning timelines, generating subtitles, and laying down basic BGM, while humans retain the parts of adjusting breathing rhythm, controlling aesthetics, adding complex effects, and final export verification. Different versions carry compatibility risks for fully automated export; AI drafts plus manual fine-tuning before export is currently the most reliable industrial-grade standard.

True business value lies in building a reusable content flywheel: today you tune the prompts and get the pipeline running, tomorrow you only need to replace the topic at the top—switch from AI to finance—and the entire production line automatically turns again, outputting a refined draft that can be published after minor manual adjustments. Control the pipeline, rather than being controlled by it.