SUPRA-TEX

SUPRA-TEX is a framework for generating material-bearing, relightable 3D assets from a single image; by decoupling visible material estimation, surface alignment, and observation-guided 3D completion, it reaches SOTA level when trained only on public datasets.

SUPRA-TEX is trained using only public datasets, yet achieves SOTA on both geometry-conditioned texturing and end-to-end asset generation, and the paper claims its results on in-the-wild images match or even surpass leading commercial systems.

In stage one, SUPRA-TEX trains an image editing model based on a multimodal large language model (MLLM) to predict illumination-separated PBR material maps under the reference viewpoint, outputting channels such as albedo, roughness, and metallic.

In stage two, SUPRA-TEX projects the predicted PBR maps onto the visible surface and converts them, through a pretrained material VAE encoder, into 'native observation latents' that share the same latent space as the generation target, in order to achieve hyper-alignment.

In stage three, SUPRA-TEX adopts an interleaved flow transformer, using the observation latents as fixed read-only anchors and alternating between updating unobserved regions and updating the entire surface, ultimately generating complete material latents.

SUPRA-TEX has a strong dependence on the material estimation accuracy of the 2D editing model; if the reference image has extreme lighting or occlusion, errors in the projection stage may propagate to the final result.

SUPRA-TEX validates a design principle: decouple the generation task from the perception task, and use mature capabilities from the 2D domain to feed back into 3D generation.

The project page for SUPRA-TEX has been made public.