11.84MB Controllable TTS Model Released: WER Unchanged, UTMOS Drops Only 0.1

Title

11.84MB Controllable TTS Model Released: WER Unchanged, UTMOS Drops Only 0.1

Meta Description

WhissleAI releases SteerTTS edge, a complete package of just 11.84MB, keeping WER at 0.96% and dropping UTMOS by only ~0.1, with 12-dimensional numeric control and 50 style tags, 19x real-time on CPU.

Article Content

Small TTS models are rapidly moving toward edge devices. WhissleAI recently released the edge version of SteerTTS—a fully packaged controllable English speech synthesis model of just 11.84 MB that keeps word error rate essentially unchanged while limiting UTMOS subjective quality loss to about 0.1.

From v1 to edge: an extreme size compression

Previously, WhissleAI released SteerTTS v1 on Hugging Face, notable for allowing direct numeric control of speech features such as speed, pitch, and pauses. This edge variant (huggingface.co/WhissleAI/steer-tts-edge) inherits the same control capabilities but compresses everything an app needs to distribute—model weights, voices, text frontend, and the Python runtime—down to 11.84 MB.

The meaning of this number deserves unpacking: it is not merely the size of the weight file, but "everything an app has to ship." For mobile or embedded scenarios, this framing is more informative than parameter count.

How the compression was achieved

The three modules each made different trade-offs:

• Acoustic model (9.25M parameters, attention + bidirectional Mamba-3 architecture) compressed to int4 using quantization-aware training (QAT);
• Vocoder (6.2M-parameter Vocos-style model) distilled from charactr/vocos-mel-24khz, also with int4 QAT;
• Text frontend rewritten in pure numpy, eliminating runtime dependencies on NLTK, g2p_en, torchaudio, or num2words.

The text frontend handling is easy to overlook but critical for edge deployment: these NLP dependencies are often large and have long installation chains, and removing them is what truly makes the whole distribution package "small and self-contained."

Quantization-Aware Training (QAT) is a technique that simulates quantization error during the training phase. Ordinary post-training quantization (PTQ) first trains the model to convergence, then directly truncates weights from fp32 to int4, often causing significant accuracy loss; QAT instead inserts "fake quantization" nodes into the forward pass during training, rounding weights to int4 granularity and then dequantizing back to floating point for computation, so the model learns to adapt to this low-precision representation during training. During backpropagation, gradients typically bypass the non-differentiable rounding operation using a straight-through estimator (STE). Models obtained this way suffer far less accuracy degradation when actually deployed as int4 than with PTQ, at the cost of requiring retraining or fine-tuning. For numerically sensitive networks like acoustic models and vocoders, QAT is a key means of preserving audio quality under aggressive compression.

Bidirectional Mamba refers to a sequence modeling architecture based on state space models (SSMs). Traditional Mamba recurses unidirectionally along the sequence with linear complexity, excelling at long sequences with fixed state during inference; bidirectional Mamba scans the sequence with both forward and backward SSMs, allowing each position to aggregate context from both sides—closer in effect to a bidirectional RNN or Transformer's global attention, but with computation and memory still growing linearly. For TTS acoustic models, bidirectional modeling helps capture long-range dependencies in prosody and phoneme coarticulation, while linear complexity fits the strict compute and memory constraints of edge devices—one reason this architecture was chosen for the 9.25M-parameter small model.

Quality ledger: intelligibility intact, slight compromise on audio quality

The author ran comparisons locally following the Open TTS Leaderboard evaluation protocol (unofficial submission). Results:

Version	Seed-TTS WER / UTMOS	CV3-Eval WER / UTMOS
v1 fp32 (no refiner)	0.96% / 4.20	—
edge	0.96% / 4.11	—

WER is identical, indicating compression did not harm intelligibility, the core metric. UTMOS dropped from 4.20 to 4.11—about 0.1 loss—which the author notes comes almost entirely from the small int4 vocoder. This attribution is instructive: if further quality improvements for the edge version are pursued, the vocoder is the highest-priority target, not the acoustic model.

Control capabilities largely preserved after quantization

Controllability is this model's core selling point versus ordinary small TTS. The author claims all 12 control dimensions—speed, loudness, pitch, pitch range, sentence-final rising intonation, pauses, tilt, breathiness, etc.—retain at least 95% of the fp32 version's control effect.

It also supports 50 style tags and inline markup, for example:

Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow> <whisper>Don't tell anyone.</whisper>

This inline syntax lets developers directly control pauses, speed, and whisper style within the text, without requiring an additional control interface.

UTMOS is an objective evaluation model for speech naturalness based on MOS prediction, trained on large amounts of human subjective rating data; given a speech sample, it outputs a predicted value close to human subjective scores (typically 1–5). It is often used in TTS research as an automatic metric requiring no human listening tests, complementary to WER (which measures intelligibility): WER only checks whether words are heard correctly, while UTMOS reflects overall perception including timbre, prosody, and noise. Seed-TTS WER refers to an evaluation benchmark proposed in the Seed-TTS paper, which computes word error rate by transcribing synthesized speech with a unified ASR model; because the evaluation process is standardized, it has been adopted by the Open TTS Leaderboard and others for cross-model comparison. Note that both are objective proxy metrics, and UTMOS may still deviate from true human listener preference; whether the 0.1 drop is noticeable in subjective perception is best confirmed with human blind tests.

Speed and limitations: a clearly positioned edge solution

In a pure CPU environment (2018 Threadripper, under load), RTF with 4 threads is 0.053—about 19x real-time; with sentence-level streaming, first-packet audio latency is about 120 ms. This latency level is already sufficient to support interactive speech applications.

The author also candidly lists limitations:

• English only, with 200 built-in voices (the full version has 2,825);
• No voice cloning, free-text prompting, or flow refiner on-device; these capabilities remain in the full model;
• Weights are dequantized to fp32 at load time, so this version saves size, not inference speed—unless operator kernels are rewritten for edge devices;
• Most emotion tags are merely style switches, with fast/anger effects being relatively weak.

The last limitation is worth noting: many small models advertise "quantization acceleration," but here the author explicitly distinguishes size benefits from speed benefits—a framing that is more helpful for engineering selection.

RTF (Real-Time Factor) is a core metric for measuring speech synthesis speed, defined as the ratio of computation time required to synthesize an audio segment to that audio's duration. RTF = 0.053 means generating 1 second of audio takes only about 0.053 seconds of computation, i.e., about 19x real-time (1/0.053 ≈ 18.9); smaller is faster, and RTF < 1 means real-time is achieved. The accompanying first-packet latency refers to the time from input text to the first playable audio segment, which is key to the experience of interactive speech applications (such as voice assistants and real-time conversation); with sentence-level streaming synthesis, the model can emit the opening audio before the entire sentence is generated, pushing perceived latency down to the hundreds of milliseconds. Only by combining these two metrics can one fully describe whether a TTS solution is "usable" on-device.

Positioning judgment from multiple perspectives

The above information comes from the author's release post on Reddit and the Hugging Face model card—single-source self-reported data. Since WER and UTMOS were both run locally under the Open TTS Leaderboard protocol and are not official leaderboard submissions, cross-model comparisons should be made with caution. However, the author has disclosed the model address and demo page (whissleai.github.io/steer-tts), so the community can reproduce and verify.

The author explicitly solicits two types of feedback: directions for improving the vocoder quality gap, and desired edge targets (ONNX, mobile, WASM). For teams needing to deploy speech synthesis on-device, this 11.84 MB solution provides a baseline worth testing.

One-line Summary

SteerTTS edge achieves controllable English TTS in an 11.84 MB full package, with WER unchanged and UTMOS down only 0.1.

Paragraph Summary

WhissleAI releases SteerTTS edge, compressing model weights, voices, text frontend, and Python runtime into 11.84 MB while retaining 12 control dimensions (speed, pitch, pauses, etc.) and 50 style tags. Compression relies on int4 quantization-aware training for the acoustic model and vocoder, plus rewriting the text frontend in pure numpy to remove heavy dependencies like NLTK. Under local evaluation per the Open TTS Leaderboard protocol, WER stays at 0.96% and UTMOS drops from 4.20 to 4.11, with the loss mainly attributed to the small int4 vocoder. On CPU with 4 threads, RTF is 0.053 and first-packet latency is about 120 ms. Limitations include English-only support, 200 voices, no on-device voice cloning, and quantization saving size rather than inference speed.