COILD: A Machine Translation Parallel Corpus and Benchmark Centered on Indic Languages

The Long-Standing Challenges of Indic Language Machine Translation

Machine translation has made remarkable strides in recent years, but this progress has been distributed unevenly across languages. For Indic languages, the lack of high-quality translation resources has long been an almost insurmountable barrier. Researchers in a recent paper highlight that machine translation for Indic languages has been held back by two persistent bottlenecks: the absence of high-quality, Indic-centric parallel corpora, and the lack of suitable evaluation benchmarks.

The problem runs even deeper when examining how existing resources are built. Most multilingual resources today are constructed using English as a pivot language — content is first translated into English, then from English into the target language. While seemingly efficient, this approach often loses the linguistic diversity, cultural nuance, and domain-specific expression that are intrinsic to Indic languages. In other words, Indic-language text translated via English as an intermediary may be grammatically correct, yet feel stilted in cultural context and real-world usage.

A parallel corpus is the core training material for machine translation — a bilingual, sentence-aligned dataset pairing source-language sentences with their corresponding target-language translations. Neural machine translation models require large amounts of such data to learn cross-lingual correspondences. For high-resource languages like English, Chinese, and French, the internet offers vast amounts of naturally occurring parallel data in the form of bilingual news articles, legal documents, and subtitles. Most Indic regional languages, however, lack large-scale digitized bilingual resources of this kind, leaving model training severely data-starved and translation quality far behind mainstream languages. This unequal distribution of resources is known in NLP as the "low-resource language" problem, and it remains one of the central challenges in multilingual AI research.

The COILD Corpus: Scale and Coverage

To address this bottleneck, the research team introduces COILD — a truly Indic-centric parallel corpus of considerable scale: over 1.16 million sentence pairs that have been human-translated and human-verified, covering 20 Indic language pairs.

These language pairs span four major language families — Indo-Aryan, Dravidian, Tibeto-Burman, and Austro-Asiatic. This cross-family coverage is particularly significant given India's extraordinary linguistic diversity: grammatical structures and expressive conventions differ enormously across these families, making it nearly impossible for any single-source corpus to adequately represent all of them.

Rather than relying on an English pivot, COILD is built entirely from original Indic-language source material, drawn from licensed corpora and spanning eight domains directly relevant to real-world applications. This means the corpus is not only large in scale, but also grounded in authentic usage contexts — reducing the cultural and contextual distortions introduced by English as an intermediary.

A Domain-Oriented Evaluation Benchmark

Having training data is only part of the equation; consistently evaluating translation quality is equally critical. COILD therefore comes paired with a domain-centric evaluation benchmark comprising 2,000 expert-verified sentences.

The value of this benchmark lies in its support for consistent multilingual and cross-lingual evaluation across Indic language pairs. In the past, different studies typically used their own test sets, making cross-study comparisons difficult. A unified, expert-reviewed benchmark enables reproducible, comparable standards for future research — a development of substantial importance for the healthy progress of the entire field.

Experimental Validation: Consistent Gains from Fine-Tuning

To validate COILD's practical effectiveness, the research team fine-tuned two representative multilingual neural machine translation models: IndicTrans2-Distilled and NLLB-200. These two models respectively represent Indic-language-specialized and general-purpose multilingual approaches.

Experimental results show consistent improvements — across different language pairs, different domains, and across both automatic metrics and human evaluation. This multi-dimensional consistency is a compelling signal that the gains are not artifacts of any particular test set, but rather general benefits arising from high-quality, Indic-centric supervised data.

Noteworthy is the dual validation through both human evaluation and automatic metrics. A common issue in machine translation is the disconnect between automatic metrics (such as BLEU) and actual translation quality. By relying on both types of evaluation, COILD's experiments strengthen the credibility of their conclusions.

BLEU (Bilingual Evaluation Understudy) is the most widely used automatic evaluation metric in machine translation. It quantifies translation quality by calculating the n-gram overlap between machine-generated translations and reference translations, with scores ranging from 0 to 100. BLEU's advantages are its computational efficiency and large-scale reproducibility, but its limitations are equally notable: it is insensitive to synonym substitution and word order variations, and sometimes fails to capture fluency and semantic accuracy. Its correlation with human judgment is particularly weak for morphologically rich languages such as those found in India. For this reason, COILD's experiments also incorporate human evaluation, where bilingual raters score translations on faithfulness and fluency — the alignment between both types of results more powerfully supports the conclusions about corpus validity.

Implications for Multilingual AI

COILD's contribution extends beyond simply providing a batch of training data. It offers a resource with both training and evaluation value for advancing multilingual machine translation in Indic languages — and for the multilingual large language models of the future.

In the era of large language models, data quality has an increasingly pronounced impact on model capability. For Indic languages that have long been in a low-resource state, a corpus exceeding one million human-verified sentence pairs with cross-family coverage has the potential to become critical infrastructure for subsequent research and product development. The philosophy it champions — moving away from English pivoting and back to native language construction — also offers valuable lessons for researchers working on other low-resource languages.

IndicTrans2-Distilled is a neural machine translation model designed specifically for Indic languages, developed by the AI4Bharat research team. It supports translation between over twenty Indic languages and English, and is currently one of the mainstream baseline models in the field of Indic language machine translation. NLLB-200 (No Language Left Behind) is a general-purpose multilingual translation model released by Meta AI in 2022, covering 200 languages worldwide, with the goal of providing foundational translation capabilities for low-resource languages. By selecting these two model types for comparative experiments, the study both examines COILD's benefits for Indic-language-specialized architectures and assesses its transfer value for general-purpose multilingual models — giving the experimental conclusions broader applicability.