FlowTranslator
A training-free in-image machine translation (IIMT) framework based on flow editing of text-to-image diffusion models. Through OCR recognition, LLM translation prompting and trajectory control, it generates translated images without requiring paired training data.
Timeline (last 90 days)
FlowTranslator is a training-free IIMT method that leverages the multilingual text generation capability of T2I diffusion models to achieve in-image text translation through flow editing.
Region Coupling Release (RCR) partially decouples the target branch noise within text regions during the early stage of the diffusion process, giving the model greater freedom to make modifications in text regions.
Coupling Expansion (CE) amplifies newly emerging editing displacements in the middle stage of the trajectory, making the editing effect more pronounced and thorough.
Under the EasyOCR read-back protocol, FlowTranslator achieved an average PRIM BLEU of 20.6 and COMET of 57.2, significantly outperforming the strong trained baseline UniTranslator.
On the real-scene photo dataset ScenePair (translated into Chinese), FlowTranslator achieved an exact match accuracy of 40.1%.
FlowTranslator's performance on ScenePair surpasses existing editing systems, some of which only handle the text rendering stage rather than the complete IIMT task.
FlowTranslator still currently depends on the accuracy of OCR and LLMs, and future work needs to improve robustness in complex scenes.
All Facts (7)
FlowTranslator is a training-free IIMT method that leverages the multilingual text generation capability of T2I diffusion models to achieve in-image text translation through flow editing.
50%UnverifiedRegion Coupling Release (RCR) partially decouples the target branch noise within text regions during the early stage of the diffusion process, giving the model greater freedom to make modifications in text regions.
50%UnverifiedCoupling Expansion (CE) amplifies newly emerging editing displacements in the middle stage of the trajectory, making the editing effect more pronounced and thorough.
50%UnverifiedUnder the EasyOCR read-back protocol, FlowTranslator achieved an average PRIM BLEU of 20.6 and COMET of 57.2, significantly outperforming the strong trained baseline UniTranslator.
50%UnverifiedOn the real-scene photo dataset ScenePair (translated into Chinese), FlowTranslator achieved an exact match accuracy of 40.1%.
50%UnverifiedFlowTranslator's performance on ScenePair surpasses existing editing systems, some of which only handle the text rendering stage rather than the complete IIMT task.
50%UnverifiedFlowTranslator still currently depends on the accuracy of OCR and LLMs, and future work needs to improve robustness in complex scenes.
50%