Rethinking CFG: Hidden Pitfalls and Corrections in TI2I Flow Matching

Introduction: When a "Standard" Component Becomes a Bottleneck

In the field of generative models, Classifier-Free Guidance (CFG) has become almost synonymous with high-quality output. From text-to-image generation to today's more complex Text-and-Image-to-Image (TI2I) flow matching tasks, CFG is regarded as an indispensable component. However, a study from an RSS source offers a subversive observation about CFG's role in TI2I modeling: during the noise stage of flow models, the CFG mechanism is actually secretly undermining editing capability, hindering instruction following and controllability.

This study does not simply reject CFG, but reveals a neglected detail—CFG's effect at different time steps is not homogeneous. The authors propose a simple correction in the early flow steps that achieves a richer alignment-preservation trade-off without additional computational cost. The effectiveness of this method has been verified on both pixel-space and latent-space TI2I flow models.

Classifier-Free Guidance (CFG) is one of the most mainstream guidance techniques in current diffusion models and flow matching models. Its basic approach is: during training, conditions (such as text) are dropped with a certain probability, so that the same network can predict both conditional and unconditional distributions; during sampling, the formula $\hat{\epsilon} = \epsilon_{\text{uncond}} + s \cdot (\epsilon_{\text{cond}} - \epsilon_{\text{uncond}})$ subtracts the two and then amplifies, where $s$ is the guidance strength. The intuition is: the difference between conditional and unconditional predictions represents the "direction carried by the condition", and amplifying this direction brings the output closer to the condition. The cost of CFG is that sampling requires two forward computations and sacrifices some diversity. In text-to-image generation, it has become almost the default means of improving text alignment, which is why it has been habitually carried over to TI2I tasks.

Core Finding: CFG's "Hidden Harm" During the Noise Stage

Counterintuitive Observation

The research team conducted systematic experiments on TI2I flow matching methods, with surprising results: when CFG is applied to the noise stage of flow models (i.e., the early flow steps), it does not improve quality as it does in pure generation tasks, but instead weakens the model's ability to follow editing instructions. This means that the CFG strength adjustment techniques we take for granted may backfire in TI2I scenarios.

Why does this happen? The key lies in the multimodal nature of TI2I tasks. With both text and image conditions present simultaneously, CFG may overly suppress the structural information of the image itself when amplifying the condition signal, causing editing results to deviate from expectations. Especially in the early stages of editing, when image structure has not yet stabilized, CFG's strong guidance introduces bias.

Flow Matching is a class of generative frameworks that share origins with diffusion models but have a more concise form. It does not define a Markov chain of progressive noising, but directly learns a time-continuous velocity field $v_\theta(x_t, t)$, transporting the noise distribution along a probability flow ODE to the data distribution. The training objective is typically to regress the conditional velocity $x_1 - x_0$ from noise to data, and sampling starts from pure noise, using numerical integration (such as the Euler method) to advance along the velocity field for several steps to obtain samples. The so-called "noise stage" or "early flow steps" refers to the first few steps of integration, where $t$ is close to 0 and the sample is still close to pure noise. In this stage, the structural information of the image has not yet taken shape, and the way the condition signal acts differs greatly from later stages—this is precisely the time window where CFG behavior becomes anomalous.

Essential Difference from Generation Tasks

In traditional text-to-image generation, CFG enhances text alignment by comparing conditional and unconditional predictions. But TI2I editing requires the model to preserve the original image content while making modifications according to instructions. This "alignment-preservation" trade-off is particularly fragile during the noise stage. The study points out that CFG's hidden harm is precisely because it breaks this balance.

TI2I (Text-and-Image-to-Image) flow matching differs from pure text-to-image generation in that the conditions are multimodal: the model receives both the source image and the editing instruction text, with the goal of making modifications according to the instruction while preserving as much as possible of the content in the source image not involved in the instruction. This creates an "alignment-preservation trade-off": text conditions drive change, while image conditions constrain against change. CFG only amplifies the text condition, equivalently unilaterally enhancing the "change" direction, without a corresponding mechanism to protect "preservation". In pure generation tasks, there is no source image to preserve, so this side effect does not manifest; in TI2I, amplifying text guidance systematically erodes the constraining power of image conditions, thereby harming the precision of instruction following and the controllability of editing.

Simple Correction: A Calibration Strategy for Early Flow Steps

An Improvement with No Extra Cost

Surprisingly, the solution is extremely simple. The authors propose correcting CFG in the early flow steps—specifically, adjusting the way CFG is applied during the noise stage so that it no longer interferes with following editing instructions. This correction requires no model retraining and adds no inference cost, being entirely completed during the sampling stage.

The core idea of this correction is: in the early steps of the flow model, reduce or reweight the influence of CFG, allowing image conditions and text conditions to merge more naturally. As the flow steps progress, standard CFG is gradually restored to maintain the quality of the final output. This dynamic adjustment achieves a finer alignment-preservation trade-off.

Dual Validation in Pixel Space and Latent Space

The study validated this method on two mainstream TI2I flow matching paradigms: pixel-space flow models and latent-space flow models. Results show that the corrected CFG significantly improves instruction following and editing controllability while maintaining high-quality generation results. This demonstrates the universality of the finding, independent of specific model architectures.

Industry Implications: Rethinking the Design of Guidance Mechanisms

This study brings important implications to the field of generative AI. As a widely adopted "default configuration", CFG's behavior can vary greatly across different tasks. For tasks like TI2I that require balancing multiple conditions, blindly applying guidance strategies from generation tasks may not be optimal.

In the future, we may need more adaptive guidance mechanisms tailored to specific tasks. For example, designing time-step-sensitive CFG scheduling strategies, or developing guidance methods that can dynamically perceive condition conflicts. This study opens the door to this direction and reminds researchers and engineers: while pursuing high-quality generation, one should not overlook the potential impact of guidance mechanisms on controllability.

Conclusion

CFG is not a master key. In TI2I flow matching, its hidden harm during the noise stage has been systematically revealed for the first time, and a simple early correction can bring significant improvement. This finding not only enhances the performance of editing tasks but also prompts us to re-examine those components taken for granted. For researchers and developers engaged in image editing and controllable generation, this is undoubtedly a turning point worth attention.

One-line Summary

Study finds CFG harms editing capability during the noise stage of TI2I flow matching, and proposes a zero-cost early correction.

Paragraph Summary

This study offers a subversive observation regarding the role of Classifier-Free Guidance in Text-and-Image-to-Image flow matching tasks: unlike text-to-image generation where CFG generally improves quality, in TI2I scenarios, applying CFG to the noise stage (early flow steps) of flow models implicitly weakens the model's ability to follow editing instructions, harming controllability. The reason lies in the multimodal condition nature of TI2I tasks—when CFG amplifies the text condition signal, it overly suppresses the structural information of the source image, breaking the "alignment-preservation" balance, which is particularly fragile in the early stage when image structure has not yet stabilized. The authors accordingly propose a minimalist correction strategy: reduce or reweight CFG's influence in the early flow steps, then gradually restore standard CFG, all without additional training or inference cost. This correction has been validated on both pixel-space and latent-space TI2I flow models, significantly improving instruction following and editing controllability while maintaining generation quality.