LLaCO
A text generation framework based on continuous diffusion models that addresses temporal compression through competition-aware semantic scheduling (SRS) and spherical noising, and includes a direction adapter and timestep-conditioned modulation.
Timeline (last 90 days)
The LLaCO framework adapts a pretrained backbone network to continuous representations through an embedding direction adapter and timestep-conditioned modulation.
LLaCO adopts a spherical noising process, formalized as geodesic interpolation toward a mask-biased endpoint, thereby eliminating norm decay and preserving accurate token competition.
LLaCO is an 8B-parameter model instance that surpasses previous continuous diffusion baselines in experiments and is comparable to discrete diffusion models of the same scale.
The limitation of LLaCO is that the experiments mainly compare against continuous diffusion baselines, and its competition with discrete diffusion models is only on par rather than surpassing.
All Facts (4)
The LLaCO framework adapts a pretrained backbone network to continuous representations through an embedding direction adapter and timestep-conditioned modulation.
50%UnverifiedLLaCO adopts a spherical noising process, formalized as geodesic interpolation toward a mask-biased endpoint, thereby eliminating norm decay and preserving accurate token competition.
50%UnverifiedLLaCO is an 8B-parameter model instance that surpasses previous continuous diffusion baselines in experiments and is comparable to discrete diffusion models of the same scale.
50%UnverifiedThe limitation of LLaCO is that the experiments mainly compare against continuous diffusion baselines, and its competition with discrete diffusion models is only on par rather than surpassing.
50%