Kitten-TTS 2 Open-Sourced: A 1.58-Bit TTS That Runs in Real Time on CPU with Voice Cloning

KittenML has just open-sourced the code and weights for Kitten-TTS 2, a cutting-edge speech synthesis model billed as "the world's first 1.58-bit TTS." Its biggest draw: no GPU required—it generates speech in real time on CPU alone, while supporting zero-shot voice cloning, 20+ languages, and emotion and accent control. For developers lacking GPU resources who still want to self-host a TTS service, this is practically an "ElevenLabs alternative"-level option.

Ternary weights: what 1.58 bits actually means

The core selling point of Kitten-TTS 2 is Ternary TTS. Each linear layer's weights in the main model take only three values: -1, 0, and +1, paired with scaling factors using a group size of 128. This ternary quantization is exactly where "1.58 bits" comes from—because log2(3) ≈ 1.58.

The model has 1.7B parameters, but since weights are compressed to ternary values, actual storage and compute overhead is far lower than a full-precision model of the same parameter count. KittenML also provides optimized ternary kernels for CPU inference, which lets it hit real-time speeds on ordinary consumer CPUs. In other words, the hardware barrier to speech synthesis has been lowered to an unprecedented point.

Why CPU real-time matters

In the past, high-quality TTS almost always required a GPU by default, making self-hosting costly with a long deployment chain. By moving inference to CPU, Kitten-TTS 2 means you can run a speech service on an ordinary server or even a laptop, with extremely low marginal cost. For budget-constrained indie developers and small teams, this is a tangible productivity shift.

Ternary quantization with grouped scaling is key to understanding 1.58 bits. Traditional quantization maps weights to more discrete levels (e.g., 256 levels for 8-bit), whereas ternary quantization keeps only three values {-1, 0, +1}, so each weight theoretically needs only log2(3) ≈ 1.58 bits. But ternary weights have limited expressiveness, so scaling factors are needed: the model divides weights into fixed-size groups (here group size 128), each group sharing a floating-point scale factor, and actual computation multiplies ternary weights by that factor to restore magnitude. This "group quantization" preserves some dynamic range while compressing storage, a common trade-off for low-bit large models.

Ternary kernels refer to compute kernels specially optimized for ternary weights. Since weights take only three values, multiplication can be reduced to addition, negation, or zeroing—avoiding floating-point multiply-add operations and greatly reducing CPU compute overhead. This is the underlying reason Kitten-TTS 2 can do real-time inference on consumer CPUs.

Voice cloning and multilingual support: replicate a voice with 5 seconds of reference audio

Kitten-TTS 2 supports zero-shot voice cloning: with less than 5 seconds of reference audio, it can clone the target timbre while preserving the original voice's expressiveness, accent, and other characteristics—no fine-tuning required. This is relatively advanced for open-source TTS.

On the multilingual front, the model supports voice cloning in 20+ languages. Even more interesting is cross-lingual cloning—you can use a Chinese reference audio clip to synthesize English speech, producing an accented voice effect. For scenarios requiring multilingual content distribution, this flexibility saves the trouble of finding a separate voice for each language.

Emotion and emphasis control

Beyond choosing a timbre, Kitten-TTS 2 also lets you "direct" the tone: you can guide the overall expressive style and add emphasis to specific words. Specific usage is documented in the project's Readme. This kind of fine-grained control is typically a paid capability of commercial TTS, and an open-source model achieving it is valuable for podcasts, audiobooks, video dubbing, and similar scenarios.

Zero-shot voice cloning means the model can imitate a speaker's timbre from a short reference audio clip without any fine-tuning or training on that target speaker. The principle is usually to have the model extract speaker features (timbre, accent, prosody, etc.) from the reference audio as conditional input, then synthesize speech for new text accordingly. Compared with traditional approaches that require training a separate model per voice, zero-shot requires no extra training cost—at deployment you simply swap the reference audio to switch voices, which is why it suits multilingual content distribution.

From V0.8 to 2: a complete architecture rewrite

According to the release notes, going from V0.8 to 2 is not a minor patch but a completely new architecture. Changes include: significantly improved speech quality, more natural listening experience, introduction of ternary weights, addition of zero-shot voice cloning, multilingual support, and emotion/emphasis control. You could say V0.8 was more of an experiment, while version 2 is a complete product built for real deployment.

The model is released as open source, with both code and weights free to use, hosted on GitHub and Hugging Face, with community discussion on KittenML's Discord. The publisher explicitly positions it as "an open-source alternative for those lacking GPU resources," emphasizing ease of self-deployment and local operation.

TTS (Text-to-Speech) systems typically include a text frontend, acoustic model, and vocoder, responsible for converting text into natural speech. In recent years, mainstream approaches are mostly neural network-based; higher audio quality often demands more compute, which is why high-quality TTS has long depended on GPUs. Kitten-TTS 2's positioning is precisely in this context—using ternary quantization and CPU-optimized kernels to bring inference costs down to what ordinary hardware can handle.

Who it's for, and what to watch out for

Kitten-TTS 2 best suits three groups: developers without GPUs who want low-cost self-hosted TTS; content creators needing multilingual voice cloning; and voice application teams with emotion control requirements.

One caveat: while 1.58-bit quantization delivers extreme efficiency, there is still room for trade-off in audio quality ceiling compared with full-precision large models. For actual results, it's advisable to do a small-sample listening test with reference audio before deciding to integrate into production. In addition, voice cloning technology itself involves copyright and personality rights issues—be sure to confirm authorization for reference audio before commercial use.

Project links: GitHub (KittenML/KittenTTS), Hugging Face (KittenML/kitten-tts-2).