[KongchangAI]
Model

WaveGen

An end-to-end text-to-speech synthesis model that generates waveforms directly from text based on a diffusion model, without intermediate representations or an external decoder. Its core design includes Spectral Forcing, a decoupled diffusion head, and a multi-scale spectral v-loss, enabling single-stage training.

Timeline (last 90 days)

Nov 10

WaveGen is a diffusion model that generates waveforms directly from text, without relying on any external decoder, duration predictor, or self-supervised semantic representation.

Unverified50%
Nov 10

The key design of WaveGen is Spectral Forcing, which has a single model learn both the model's internal spectral trajectory and the target waveform trajectory simultaneously, with the spectral trajectory serving as internal scaffolding to help stabilize the diffusion process under high compression ratios.

Unverified50%
Nov 10

WaveGen adopts a disentangled diffusion heads architecture and a multiscale spectral v-loss training objective; text-speech alignment is done inside the model, and the duration predictor is removed.

Unverified50%
Nov 10

WaveGen scales the model from 0.1B parameters to 0.6B parameters, validating the scalability of end-to-end TTS.

Unverified50%
Nov 10

On the Emilia training dataset, WaveGen reports state-of-the-art performance.

Unverified50%
Nov 10

On the Seed-en benchmark, WaveGen achieves a word error rate (WER) of 1.47 and a speaker similarity (SIM) of 0.75.

Unverified50%
Nov 10

WaveGen's training pipeline is a single architecture, open-source data, and one-stage training, requiring no auxiliary optimization methods such as REPA.

Unverified50%
Nov 10

The WaveGen project is open-sourced on GitHub (sh-lee-prml/WaveGen), with both code and checkpoints released.

Unverified50%
Nov 10

The WaveGen internal spectral trajectory is essentially still an implicit intermediate representation, except that it is no longer solidified by external modules, and its interpretability and intervenability are currently open questions.

Unverified50%

All Facts (9)

Unverified

The WaveGen internal spectral trajectory is essentially still an implicit intermediate representation, except that it is no longer solidified by external modules, and its interpretability and intervenability are currently open questions.

50%
Unverified

The key design of WaveGen is Spectral Forcing, which has a single model learn both the model's internal spectral trajectory and the target waveform trajectory simultaneously, with the spectral trajectory serving as internal scaffolding to help stabilize the diffusion process under high compression ratios.

50%
Unverified

WaveGen adopts a disentangled diffusion heads architecture and a multiscale spectral v-loss training objective; text-speech alignment is done inside the model, and the duration predictor is removed.

50%
Unverified

WaveGen scales the model from 0.1B parameters to 0.6B parameters, validating the scalability of end-to-end TTS.

50%
Unverified

On the Emilia training dataset, WaveGen reports state-of-the-art performance.

50%
Unverified

On the Seed-en benchmark, WaveGen achieves a word error rate (WER) of 1.47 and a speaker similarity (SIM) of 0.75.

50%
Unverified

WaveGen's training pipeline is a single architecture, open-source data, and one-stage training, requiring no auxiliary optimization methods such as REPA.

50%
Unverified

WaveGen is a diffusion model that generates waveforms directly from text, without relying on any external decoder, duration predictor, or self-supervised semantic representation.

50%
Unverified

The WaveGen project is open-sourced on GitHub (sh-lee-prml/WaveGen), with both code and checkpoints released.

50%

Source Articles