TI2I
A multimodal conditional generation task in which the model receives both a source image and an editing instruction text. The goal is to modify the image according to the instruction while preserving content in the source image not covered by the instruction. The core challenge is the alignment-preservation trade-off.
Timeline (last 90 days)
The conditions of the TI2I task are multimodal: the model simultaneously receives the source image and the editing instruction text, forming an 'alignment-preservation' trade-off, where the text condition drives change and the image condition constrains what stays unchanged.
In TI2I, CFG only amplifies on the text condition, which is equivalent to unilaterally strengthening the 'change' direction without any mechanism to protect 'preservation', thereby systematically eroding the constraining force of the image condition.
For tasks like TI2I that need to balance multiple conditions, blindly applying the guidance strategies of generation tasks may not be the optimal solution.
In pure text-to-image generation tasks there is no source image that needs to be preserved, so the side effects of CFG amplifying text guidance do not manifest; in TI2I, however, these side effects harm instruction-following precision and editing controllability.
All Facts (4)
The conditions of the TI2I task are multimodal: the model simultaneously receives the source image and the editing instruction text, forming an 'alignment-preservation' trade-off, where the text condition drives change and the image condition constrains what stays unchanged.
50%UnverifiedIn TI2I, CFG only amplifies on the text condition, which is equivalent to unilaterally strengthening the 'change' direction without any mechanism to protect 'preservation', thereby systematically eroding the constraining force of the image condition.
50%UnverifiedFor tasks like TI2I that need to balance multiple conditions, blindly applying the guidance strategies of generation tasks may not be the optimal solution.
50%UnverifiedIn pure text-to-image generation tasks there is no source image that needs to be preserved, so the side effects of CFG amplifying text guidance do not manifest; in TI2I, however, these side effects harm instruction-following precision and editing controllability.
50%