ProtoDiffusion
A diffusion model for multi-layer graphic design generation that compresses representations into HW+N tokens via shared spatial prototypes and per-layer candidate vectors, supports image- and text-conditioned generation, and can infer layer layouts.
Timeline (last 90 days)
ProtoDiffusion proposes a multi-layer image generation representation: all layers share a single spatial prototype, and each layer retains only one candidate vector, thereby compressing the representation of an N-layer design to HW+N tokens (before backbone-specific packing), whereas a full-grid scheme requires NHW tokens.
ProtoDiffusion performs joint denoising in a compact latent space, simultaneously denoising the shared prototype and a fixed number of candidate slots, with conditions being either images or text and bounding boxes serving as optional layout guidance.
ProtoDiffusion's candidate slots can be left empty, and the model decides on its own which slots are empty, so the number of layers need not be provided as input.
ProtoDiffusion can also infer layer layout when no bounding boxes are provided, with bounding boxes acting only as optional constraints rather than required inputs.
The same ProtoDiffusion architecture supports both image-to-design and text-to-design tasks, without needing separate networks designed for the two directions.
On the LayerD evaluation set, ProtoDiffusion's image-to-design RGB L1 score is 0.040 with bounding boxes and 0.057 without bounding boxes.
On the LayerD evaluation set, ProtoDiffusion achieves an alpha soft IoU of 0.850 under box guidance, comparable to the strongest fine-tuned baseline of 0.871.
In the text-to-design direction on LayerD, ProtoDiffusion achieves an FID of 66.41 on merged images, compared with 86.84 for FLUX.2 [klein].
ProtoDiffusion achieves a text accuracy score of 0.860 in the text-to-design direction.
ProtoDiffusion is 19.1 times faster than the same-backbone 'generate-then-decompose' cascade scheme, with the speed gap mainly stemming from eliminating the layer decomposition step.
3 more timeline events
All Facts (13)
ProtoDiffusion proposes a multi-layer image generation representation: all layers share a single spatial prototype, and each layer retains only one candidate vector, thereby compressing the representation of an N-layer design to HW+N tokens (before backbone-specific packing), whereas a full-grid scheme requires NHW tokens.
50%UnverifiedProtoDiffusion performs joint denoising in a compact latent space, simultaneously denoising the shared prototype and a fixed number of candidate slots, with conditions being either images or text and bounding boxes serving as optional layout guidance.
50%UnverifiedProtoDiffusion's candidate slots can be left empty, and the model decides on its own which slots are empty, so the number of layers need not be provided as input.
50%UnverifiedProtoDiffusion can also infer layer layout when no bounding boxes are provided, with bounding boxes acting only as optional constraints rather than required inputs.
50%UnverifiedThe same ProtoDiffusion architecture supports both image-to-design and text-to-design tasks, without needing separate networks designed for the two directions.
50%UnverifiedOn the LayerD evaluation set, ProtoDiffusion's image-to-design RGB L1 score is 0.040 with bounding boxes and 0.057 without bounding boxes.
50%UnverifiedOn the LayerD evaluation set, ProtoDiffusion achieves an alpha soft IoU of 0.850 under box guidance, comparable to the strongest fine-tuned baseline of 0.871.
50%UnverifiedIn the text-to-design direction on LayerD, ProtoDiffusion achieves an FID of 66.41 on merged images, compared with 86.84 for FLUX.2 [klein].
50%UnverifiedProtoDiffusion achieves a text accuracy score of 0.860 in the text-to-design direction.
50%UnverifiedProtoDiffusion is 19.1 times faster than the same-backbone 'generate-then-decompose' cascade scheme, with the speed gap mainly stemming from eliminating the layer decomposition step.
50%UnverifiedExisting cascade-style multi-layer generation schemes (generate the whole image first, then decompose layers) accumulate errors across two stages, whereas ProtoDiffusion jointly models all layers within a single model, with spatial alignment between layers naturally constrained by the shared prototype.
50%UnverifiedProtoDiffusion's shared-prototype assumption posits that all layers share the same set of spatial structure priors; for designs with greatly differing spatial distributions across layers, whether this sharing is sufficient still requires validation in more scenarios.
50%UnverifiedA fixed number of candidate slots means that designs with an extremely large number of layers must rely on slot reuse or expansion strategies.
50%