SCAIL 2 Deep Dive: Character Replacement, Physics Simulation & Object Permanence Performance Analysis

A Stress Test That Exceeded Expectations

Demos in the video generation AI space tend to be monotonous—either single-character dance clips or various physics jitter effects. To verify SCAIL 2's true capabilities, one developer chose a batch of complex scenarios "it shouldn't be able to handle well" for systematic testing, covering character replacement, complex motion, prop substitution, physics simulation, novel interactions, object permanence, relighting, and 2D motion transfer.

The results were surprising: SCAIL 2 delivered excellent results across nearly every scenario. This article distills the most noteworthy findings from this test, along with the workflow details behind them.

Character Replacement: SCAIL 2's Strongest Use Case

The tester concluded that character replacement is SCAIL 2's most outstanding capability. But achieving quality results hinges on proper reference image preparation.

Core Techniques for Reference Image Preparation

The key technique is aligning the reference image as closely as possible with the driving video's starting frame. The tester recommends using Flux Klein 9B or Krea 2 Identity Edit LoRA to edit the video's first frame, replacing the person in it with the target character so the reference image roughly matches the driving video's pose and composition from the start.

Flux Klein 9B is a variant of the Flux series image generation models from Black Forest Labs, specializing in identity-preserving image editing—it can precisely replace a person's identity while maintaining the overall composition and background. Krea 2 Identity Edit LoRA is based on LoRA (Low-Rank Adaptation) technology, a lightweight fine-tuning method that efficiently acquires specific capabilities by injecting low-rank decomposition matrices into the pre-trained model's weight matrices, typically requiring only 0.1%-1% of the original model's parameters to train. The Identity Edit LoRA is specifically optimized for facial identity replacement scenarios, naturally blending a target person's facial features into existing images while preserving the original pose and lighting conditions.

"As long as you do this, the results are excellent. Having a clean starting frame helps a lot too."

This finding reveals a universal principle: video generation model output quality is often highly dependent on the quality and alignment of the starting frame. Rather than obsessing over parameters during generation, it's better to invest effort in the input stage.

Object Permanence: Stability Beyond Expectations

Object Permanence is a classic challenge in video generation—when an object leaves the frame and returns, can the model maintain its consistency?

This concept originally comes from developmental psychologist Piaget's research, describing the human cognitive ability to understand that "objects continue to exist even when they're not visible." In video generation AI, this problem is particularly tricky: most diffusion models use temporal attention mechanism-based architectures that need to maintain "memory" of an object's properties during frames where it has left the field of view. When the number of occluded or off-screen frames becomes too large, the attention mechanism's effective receptive field is limited, and the model tends to "forget" object features, causing deformation or color shifts when the object reappears.

In a car clip, the vehicle completely exited the frame and then drove back in, maintaining high consistency throughout the entire process. The tester originally expected it to "turn into a mess," but the actual performance was remarkably stable.

"Not perfect, but not bad either."

This capability is particularly important for narrative video generation, indicating the model has some degree of "memory" and spatial understanding of the scene, rather than isolated frame-by-frame generation. SCAIL 2's strong performance in this area suggests it may employ some form of long-range memory mechanism or global conditioning injection strategy, enabling object property information to persist across extended time spans.

Generating Novel Actions: Reasonable "Imagination"

SCAIL 2 demonstrated impressive action generation capabilities—it can generate actions that never appeared in the driving video.

In the novel interaction test, the tester replaced himself with a live-action version of "Zuko" (a character from Avatar: The Last Airbender), and fire shot from the fists in believable arcs—despite the driving footage containing no fire data whatsoever. The model replicated the underlying motion and added contextually appropriate embellishments. Similar performance was observed with clothing, hair, and other elements.

This demonstrates that SCAIL 2 doesn't mechanically copy movements but understands the essence of motion, performing reasonable completion and extension on that basis. The model can infer what visual effects a character "should" produce based on their identity attributes and contextual semantics—a capability that goes beyond simple motion transfer into the realm of semantically-driven content generation.

Physics Simulation: The Biggest Surprise of This Test

The most surprising discovery in this test was SCAIL 2's physics simulation performance.

The tester replaced a flower with a wine glass, and the results:

Hand tracking remained locked, with stable gripping motion
Liquid in the glass sloshed correctly with movement
Due to the glass being transparent, the background underwent believable refraction and distortion through the water surface

Most critically—nothing in the driving clip told the model to do any of this. This means the model has internalized a considerable degree of physical common sense, capable of autonomously inferring complex phenomena like transparent materials and fluid dynamics.

This physics simulation capability belongs to what's known as "Emergent Capability" in large-scale video generation models—not implemented through explicit physics engines or rule-based programming, but implicitly learned from the statistical patterns of the physical world during training on massive video datasets. When models are large enough and training data is sufficiently rich, they can extract physical patterns from data, such as liquid sloshing with acceleration changes and light refracting when passing through transparent media. This is analogous to how large language models exhibit emergent reasoning abilities at sufficient scale—physical understanding is encoded in the model's deep feature representations, enabling it to automatically generate physically plausible visual effects based on object material properties (such as transparency, fluidity, rigidity).

Weakness: Text Rendering Remains a Clear Shortcoming

Of course, SCAIL 2 isn't perfect. The most obvious weakness in testing was text rendering.

In a character replacement clip "turning himself into an old man," the text on speed limit signs in the background became blurry and illegible. The tester therefore recommends: for best results, avoid scenes containing text.

Text rendering remains a stubborn weakness of diffusion models because text has highly discrete and precision-demanding characteristics. Most elements in natural images—faces, landscapes, objects—tolerate a degree of blur and variation without affecting perceptual quality, but even a single pixel of deviation in text can render characters unrecognizable. The diffusion model's denoising process is fundamentally a probabilistic progressive refinement, better suited to generating continuous, smooth visual features rather than structured information like letters that require precise stroke positions and spacing. Additionally, text occupies an extremely small proportion of total image area in training data, making it difficult for the model to receive sufficient gradient signals in these local regions. While recent research like GlyphDraw and TextDiffuser has attempted to improve this through additional glyph conditioning injection, text in video generation scenarios also needs to remain consistent across frames, multiplying the difficulty. SCAIL 2 is no exception.

Workflow Configuration: Mix Studio Open-Source Interface Based on ComfyUI

All tests were completed in Mix Studio—a free, open-source local interface developed by the tester, running on top of ComfyUI (GitHub project page).

ComfyUI is an open-source workflow editor with a node-based graphical interface, created by developer comfyanonymous. Unlike traditional WebUI interfaces, ComfyUI breaks down the entire image/video generation pipeline into freely connectable functional nodes (such as model loading, sampler configuration, VAE decoding, etc.), allowing users to assemble arbitrarily complex generation pipelines like building circuits. The advantage of this architecture lies in its extreme flexibility and reproducibility—any workflow can be exported as a JSON file and precisely reproduced. Mix Studio builds on top of ComfyUI, providing a friendlier interaction layer for video models like SCAIL 2, lowering the learning curve of node editing.

The basic operational flow is:

Click the Edit tab to edit the image
Press "use as first frame" to set the edited image as the video's first frame
Set the video mode to SCAIL 2

On the hardware side, testing was completed on a Dell Pro Precision T2 workstation equipped with an NVIDIA RTX 6000 Pro, with each generation taking approximately 2-3 minutes. The RTX 6000 Pro features 48GB of GDDR7 VRAM, a professional-grade GPU whose large memory capacity is particularly critical for video generation models—video models need to process multiple frames simultaneously, with VRAM demands far exceeding single-frame image generation models. The 48GB capacity ensures model weights, intermediate activations, and multi-frame tensors can reside in VRAM simultaneously without frequent offloading.

Summary: SCAIL 2's Actual Capability Boundaries

From this systematic test, SCAIL 2 demonstrated capabilities exceeding expectations in character replacement, object permanence, action generation, and physics simulation. The physical details that were never explicitly specified—such as transparent material refraction and fluid sloshing—particularly reflect the model's deep understanding of the world.

While text rendering remains a clear weakness, for creators, paired with the proper reference image preparation workflow, SCAIL 2 can already handle quite complex video generation tasks. This perhaps signals that video generation AI is moving from the "flashy demo" stage toward genuinely practical content creation tools—models are no longer mere pixel-level pattern matchers but are gradually becoming creative engines that understand physical laws, possess spatial memory, and can perform semantic reasoning.