QuanLing Framework Cross-Branch Validation: Quantifying Linguistic Distance in the Western Romance Languages

Quantifying Linguistic Distance: A Long-Standing Challenge

In quantitative linguistics, precisely measuring the distance between closely related languages has long been a core challenge. Such languages are highly similar to one another, and traditional classification methods often struggle to capture their subtle differences. A recent paper published on arXiv (arXiv:2610.08851v1) proposes a new approach — the QuanLing framework (Quantitative Linguistics via Pretrained Language Models) — which leverages pretrained language models to quantify linguistic distance.

QuanLing is not an entirely new concept. In prior work, the research team had already applied this framework to the North Germanic branch (Danish, Norwegian Bokmål, and Swedish) and completed initial validation. The core contribution of this paper is extending the methodology to another language branch — the Western Romance languages — in order to test its cross-branch generalizability.

Three Core Metrics: From Embedding Distance to Mutual Intelligibility

The QuanLing methodology is built on a family of metrics that combines linguistic distance measurement with linguistic property analysis.

Sentence Embedding Distance

The study uses the LaBSE (Language-agnostic BERT Sentence Embedding) model to compute distances between parallel sentences from different languages in vector space. The closer the distance, the more similar the two languages are at the level of semantic expression.

LaBSE (Language-agnostic BERT Sentence Embeddings) was introduced by Google in 2020 as a sentence encoding model specifically designed for cross-lingual semantic alignment, supporting 109 languages. Unlike standard multilingual models, LaBSE is explicitly trained to optimize "language agnosticism": regardless of which language a given semantic content is expressed in, the model should map it to nearby positions in vector space. Its training data includes large amounts of bilingual parallel corpora, and it uses a contrastive learning objective to pull translation pairs closer together while pushing non-translation pairs apart. This design makes it particularly well-suited as a tool for measuring linguistic distance — the average distance between two languages in LaBSE space can be interpreted as "structural difference at the level of semantic expression," rather than purely lexical or grammatical difference.

Tokenization Fragmentation Rate

The team uses four monolingual BERT tokenizers to measure the tokenization fragmentation rate. This metric reflects how fragmented a language becomes when processed by a tokenizer trained on another language — the degree of fragmentation reveals, in a sense, the lexical-level distance between the two languages.

BERT-family models universally use the WordPiece tokenization algorithm: if a word appears in the training vocabulary, it is kept as a single token; if not, it is split into smaller subword units, which are typically marked with a "##" prefix. When a tokenizer trained on language A processes text in language B, the lower the vocabulary overlap between A and B, the higher the fragmentation rate. Thus, the fragmentation rate essentially measures the degree of overlap in written vocabulary between two languages — for genetically related languages that share many roots and spellings (such as Portuguese and Spanish), the mutual fragmentation rate should be relatively low, and vice versa. This metric requires no semantic understanding and provides a complementary distance dimension purely from a string-pattern perspective.

Masked Language Model Mutual Intelligibility

The third dimension leverages mBERT's masked language model (MLM) prediction capability to measure "mutual intelligibility" between languages. A model's accuracy in predicting tokens of one language given the context of another can serve as a proxy for the degree of mutual comprehension.

The Western Romance experiment covered French, Portuguese, Spanish, and Italian, using English as an anchor to form quadruplet structures, with 150 sets of parallel sentences across the four languages. To ensure comparability, the full family of metrics and aggregation protocols remained consistent with the North Germanic study, with only minor adaptations for the four-language setup.

Key Findings: Who Is Closest, Who Is Most Distant

The experimental results paint a clear map of linguistic distances.

Portuguese and Spanish are the closest pair among the four languages, with a LaBSE distance of just 0.0229 — entirely consistent with their close historical and geographical ties. French and Italian are the most distant pair, with a LaBSE distance of 0.0338.

Cross-model consistency is an important validation point in this study. Among the six language pairings, LaBSE's ranking results agreed with mBERT's rankings in 4 out of 6 cases, providing empirical support for QuanLing's cross-model robustness. Two independent models arriving at similar conclusions suggests that the measured distances are not an artifact of any single model.

Cross-Branch Comparison: Larger Absolute Span, Similar Relative Ratio

Comparing the Western Romance results with the earlier North Germanic findings reveals an intriguing pattern.

The Western Romance languages exhibit a larger absolute distance span (0.011, compared to 0.008 for North Germanic), yet a fairly similar relative ratio (1.48 vs. 1.67). The researchers argue this is consistent with the Western Romance languages' longer divergence time — the longer languages have been separated, the greater their absolute differences naturally become, while the relative relationships within the branch remain of comparable magnitude.

The significance of this finding is that QuanLing captures not merely surface-level distance values, but something akin to the underlying tempo of linguistic evolution.

The Special Case of French: Decoupled Orthography and Phonology

One detail worth examining closely is French's performance on MLM prediction. French exhibits noticeably higher masked language model predictability, with a top-1 accuracy of 36.12%, compared to just 29.28% for Italian.

The study attributes this difference to French's characteristic orthography–phonology decoupling. French written forms and actual pronunciation diverge extensively (for example, many word-final letters are silent), and this highly regularized yet phonologically detached spelling system actually makes it easier for text-based language models to predict — because the written patterns themselves follow more predictable regularities. This observation provides a concrete and compelling case study for the linguistic property analysis dimension of the framework.

French orthography–phonology decoupling is particularly pronounced among the Romance languages. Typical examples include: word-final consonant letters are silent in most cases ("temps" is pronounced /tɑ̃/, with the final -mps entirely mute); a single pronunciation can correspond to multiple spellings (/o/ can be written as o, au, eau, or ô); and extensive liaison rules further separate written word forms from actual spoken pronunciation. This characteristic emerged during the evolution from Middle French to Modern French: the writing system tended to preserve etymological spellings, while spoken pronunciation continued to simplify and evolve, causing the two to gradually diverge. For language models pretrained on text, the high regularity and morphological stability of written French actually creates stronger n-gram prediction signals, making the masked prediction task easier to accomplish in French.

Implications for Framework Generalizability

From North Germanic to Western Romance, QuanLing has completed a cross-branch validation. The same family of metrics and the same aggregation protocols have produced internally consistent and interpretable results across two language branches with different origins and divergent evolutionary paths.

This provides evidence for QuanLing's generalizability that goes beyond a single language branch. For quantitative linguistics, it suggests that pretrained language models have the potential to serve as a universal, cross-family tool for measuring linguistic distance, without requiring bespoke methods for each language family. Of course, validation on just two branches is not sufficient to claim full universality — future extensions to additional language families (such as Slavic, Sino-Tibetan, etc.) will be essential to truly test the boundaries of this framework.