REPA
A representation alignment method that uses a pretrained visual encoder to supervise intermediate layers of a diffusion backbone through an MLP projection head. The projection head is discarded after training in order to accelerate convergence.
Timeline (last 90 days)
The paper "Representation Conditioning for Generation" proposes the RECON method, which, without adding any parameters, speeds up diffusion Transformer training convergence to 63x that of SiT and 25x that of REPA.
REPA uses a pretrained visual encoder to supervise intermediate layers of the diffusion backbone through an MLP projection head; after training is complete the projection head is discarded, adding no inference-time computation.
REPA's alignment loss acts only on the early layers, while the later layers are supervised only by the velocity prediction loss and lack a mechanism for transmitting semantic information, which is REPA's structural flaw.
REPA handles early-layer semantic alignment, while RECON handles later-layer semantic conditioning; the two are complementary rather than conflicting.
The difference between REPA and RECON is that REPA is a soft constraint at the loss level (making backbone features fit encoder features), while RECON is hard information at the input level (feeding encoder features directly into the backbone as input).
The work of REPA and RECON shows that the semantic information learned by the early layers of a diffusion Transformer may not automatically flow to the later layers, and that designing supervision signals hierarchically and in stages may be an underestimated direction for improving training efficiency.
All Facts (6)
The paper "Representation Conditioning for Generation" proposes the RECON method, which, without adding any parameters, speeds up diffusion Transformer training convergence to 63x that of SiT and 25x that of REPA.
50%UnverifiedREPA uses a pretrained visual encoder to supervise intermediate layers of the diffusion backbone through an MLP projection head; after training is complete the projection head is discarded, adding no inference-time computation.
50%UnverifiedREPA's alignment loss acts only on the early layers, while the later layers are supervised only by the velocity prediction loss and lack a mechanism for transmitting semantic information, which is REPA's structural flaw.
50%UnverifiedREPA handles early-layer semantic alignment, while RECON handles later-layer semantic conditioning; the two are complementary rather than conflicting.
50%UnverifiedThe difference between REPA and RECON is that REPA is a soft constraint at the loss level (making backbone features fit encoder features), while RECON is hard information at the input level (feeding encoder features directly into the backbone as input).
50%UnverifiedThe work of REPA and RECON shows that the semantic information learned by the early layers of a diffusion Transformer may not automatically flow to the later layers, and that designing supervision signals hierarchically and in stages may be an underestimated direction for improving training efficiency.
50%