RECON

The paper "Representation Conditioning for Generation" proposes the RECON method, which, without adding any parameters, speeds up diffusion Transformer training convergence to 63x that of SiT and 25x that of REPA.

RECON's core idea is to feed the tokens output by a pretrained visual encoder as conditioning signals directly into the later layers of the diffusion model during training.

RECON introduces no additional parameters, so engineering deployment has almost no extra cost.

REPA handles early-layer semantic alignment, while RECON handles later-layer semantic conditioning; the two are complementary rather than conflicting.

The difference between REPA and RECON is that REPA is a soft constraint at the loss level (making backbone features fit encoder features), while RECON is hard information at the input level (feeding encoder features directly into the backbone as input).

The work of REPA and RECON shows that the semantic information learned by the early layers of a diffusion Transformer may not automatically flow to the later layers, and that designing supervision signals hierarchically and in stages may be an underestimated direction for improving training efficiency.

The extent to which RECON's effectiveness depends on the quality of the pretrained visual encoder and on the fidelity of the MLP head as a proxy during inference still requires further verification.

The RECON paper's code has been open-sourced on GitHub at github.com/saeidrazavi/RECON.

RECON's design of reusing the MLP head as a proxy during inference introduces a new source of error, namely proxy fidelity, and at the same time has general reference value in scenarios such as conditional generation and controllable generation.