CVSS-X: A Large-Scale Speech Translation Corpus Spanning 28 Languages

Speech-to-Speech Translation (S2ST) has long been constrained by the scarcity of high-quality multilingual data. The CVSS-X corpus was developed specifically to address this gap. As an extended version of the classic CVSS dataset, it reverses the translation direction — shifting the focus from "translating multiple languages into English" to "translating English into multiple languages" — providing a solid data foundation for bidirectional and multilingual speech translation research.

From CVSS to CVSS-X: Reversing the Translation Direction

The original CVSS corpus supports translation from 21 languages into English, a design that naturally favors English-as-target scenarios. In practice, however, translation needs are often bidirectional — users don't just need foreign languages converted into English; they also need English content distributed across the world's many languages.

CVSS-X fills precisely this gap. It enables translation from English into 28 target languages spanning 12 language families, covering a typologically diverse range of linguistic systems. When combined with the original CVSS, researchers can build truly bidirectional, multilingual speech translation systems — no longer confined to single-direction experiments.

Scale: 16,000+ Hours of Synthetic Speech

The sheer scale of CVSS-X is its most striking feature. Each language contains approximately 240,000 parallel speech pairs, with a total duration exceeding 16,000 hours — eight times the size of the original CVSS.

This volume is critical for training modern end-to-end speech translation models, which typically require massive amounts of aligned data to learn cross-lingual acoustic and semantic mappings. Previous public datasets have consistently fallen short in both language coverage and total duration. CVSS-X addresses this in one stroke through large-scale synthesis, providing balanced data across numerous language families and giving even low-resource languages relatively sufficient training material.

Two Variants: Standard Voice and Cross-Lingual Voice Cloning

CVSS-X offers two versions to accommodate different research scenarios:

CVSS-X-C: Standard Voice Variant

This variant provides two canonical voices per language with a consistent, stable style. It is well-suited as a baseline for evaluating translation quality, reducing the interference of acoustic variables on model training.

CVSS-X-T: Cross-Lingual Voice Cloning Variant

This variant incorporates cross-lingual voice cloning technology, allowing the same speaker's vocal characteristics to remain consistent across different languages. This is particularly valuable for research into "speaker-identity-preserving speech translation" — an ideal speech translation system should not only translate content but also retain the original speaker's voice characteristics.

Notably, both variants are fully synthetically generated, which reduces data collection costs and sidesteps the privacy and copyright concerns associated with real human speech recordings.

Translation Quality and Language Coverage

Evaluation results show that CVSS-X achieves translation quality comparable to the original CVSS, with consistent performance across typologically diverse languages. This is especially significant — a common pitfall in multilingual datasets is that resource-rich languages (such as Western European languages) perform well while low-resource or structurally distinct languages see sharp quality drops. CVSS-X's stability across language families suggests that its synthesis pipeline generalizes well.

With coverage spanning 12 language families and 28 target languages, this corpus can support comparative studies of linguistic universals and particulars, and provides more balanced training resources for voice products targeting a global user base.

Open Access and Licensing

The research team has taken an open approach. The associated code is publicly available on GitHub (github.com/ErmisAI/XVSS-X), while the dataset is hosted on Hugging Face (huggingface.co/datasets/lgris/XVSS-X) under the CC-BY-NC 4.0 license.

It is worth noting that CC-BY-NC permits free use and distribution for non-commercial purposes, but commercial applications require additional consideration. For academic researchers, this license provides ample room for use; teams looking to incorporate it into commercial products should pay close attention to the license restrictions.

Significance and Outlook

The release of CVSS-X is more than a dataset expansion — it represents a meaningful step toward truly multilingual, bidirectional speech translation capabilities. By addressing data scarcity through synthetic means and substantially broadening language coverage without sacrificing quality, it provides the infrastructure needed to build universally accessible cross-lingual speech communication tools.

As large speech models continue to evolve, large-scale, multi-family, openly licensed corpora like CVSS-X are likely to become important benchmarks for training and evaluating the next generation of speech translation systems.