Cross-Lingual Concept Subspaces: An Empirical Study of Reading, Transfer, and Erasure

Bengali idiom study: cross-lingual concept sharing is estimator-sensitive; reading, transfer, erasure need separate evaluation.
Using six frozen encoders and 10,359 Bengali idioms, a study tests cross-lingual concept subspaces. Class-mean directions show consistent coarse sharing across all 12 tests, and English readers transfer in 8 of 12. Yet reading is not erasure: English-fitted LEACE removes <1pp of Bengalimeter accuracy, Bengali-fitted 4-11pp. The study urges separate evaluation of reading, transfer, erasure, and overlap.
Cross-lingual concept analysis often assumes that different languages share some kind of semantic subspace, but a study on Bengali idioms reveals that this sharing is "estimator-sensitive"—whether a subspace can be read, whether a classifier can be transferred, and whether a concept can be erased are three different things. Using six frozen encoders, 10,359 Bengali idiom items, and eight input conditions, the study systematically examined the reading, transfer, and erasure of cross-lingual concept subspaces.

Study Design and Core Findings
The study focuses on Bengali idioms because their meaning often cannot be derived literally, making them an ideal probe for testing cross-lingual concept sharing. The authors used six frozen encoders to extract representations under eight input conditions and evaluated them using metaphor categories and sentiment labels.
The core conclusion is that class-mean directions provide the most consistent evidence of coarse-grained sharing. In post-hoc analyses, class-mean directions outperformed random subspaces, label permutations, and covariance-matched null hypotheses in balanced accuracy across all 12 model-label tests. A prospective internal re-split supported idiom-specific sharing in 10 of 12 tests, covering five encoders.
This indicates that cross-lingual signal exists, at least for coarse-grained metaphor categories. But the signal is limited in strength: the original confirmatory logistic regression family rejected both planned null hypotheses in 4 of 12 tests, with English directions exceeding the majority-class baseline by only a few percentage points (3.0–7.3).

Frozen encoders refer to using the intermediate-layer outputs of pretrained language models (such as XLM-R, mBERT, etc.) directly as fixed features in multilingual representation research, without further fine-tuning for downstream tasks. The benefit is that it eliminates interference from parameter updates during fine-tuning, so that representation differences across languages and input conditions stem entirely from the encoder itself and the input mapping. Researchers typically extract representations from multiple layers and then select the best layer on a validation set to observe how cross-lingual signal is distributed within the model. The frozen strategy also facilitates reproducibility and comparison across encoders, but the trade-off is that representations may not be optimized for the target task.
Transfer Depends on Input Mapping and Regularization
Classifier transfer is another dimension. The study found that English-trained class-mean readers outperformed both zero-shot null hypotheses in 8 of 12 discovery tests, and that stronger source-selection penalties improved transfer through logistic regression and LEACE directions. These readers used layers selected on the Bengali validation set and label-free target preprocessing.
Input controls also showed that explicit annotations exhibited stronger coarse-class sharing than idiom counterparts. This means that cross-lingual transfer is easier when the input mapping is more "transparent."
LEACE (Least-squares Concept Erasure) is a linear concept erasure method that finds a minimal linear transformation such that a linear classifier for the target concept cannot predict that concept from the representation, while preserving other information as much as possible. Its core is a closed-form solution based on the inverse of the within-class covariance matrix, which whitens the representation and then projects out the concept direction. LEACE is commonly used for fairness debiasing and concept removal. In cross-lingual settings, if LEACE is fitted on the source language and then applied to the target language, the erasure effectiveness depends on the degree of geometric alignment between the representation distributions of the two languages, which can lead to asymmetry.
INLP (Iterative Nullspace Projection) is another linear concept erasure method that iteratively trains a linear classifier and projects onto its null space to progressively remove concept information. Unlike LEACE's one-shot closed-form solution, INLP repeats the "train classifier—project" process until the classifier can no longer predict the concept. INLP also assumes that concept directions are linearly separable; in cross-lingual applications, erasure effectiveness is reduced if the concept directions in the source and target languages are inconsistent. Both INLP and LEACE are post-hoc linear methods that are computationally efficient but can only remove linearly decodable information.
Reading Is Not Erasure: Asymmetry Is Pronounced
The most important caution from the study is that reading a concept does not mean it can be effectively erased. English-fitted LEACE removed less than one percentage point of Bengali metaphor category accuracy, whereas Bengali-fitted LEACE removed 4–11 percentage points; INLP exhibits a similar asymmetry.
This asymmetry can be explained by mode-filtering analysis, covariance-weighted input mapping, and a bivariate Gaussian overlap bound. In other words, the effectiveness of erasure operations depends heavily on the geometric relationship between the fitting language and the target language.
Implications for Cross-Lingual Representation Research
The study supports an "estimator-sensitive" interpretation of cross-lingual concept sharing and calls for separately evaluating reading, classifier transfer, erasure, and overlap. Multi-class ordering, as an empirical diagnostic, was observed in 11 of 12 cells.
For practitioners working on multilingual model interpretability and fairness, the takeaway is direct: do not assume that a concept erasure method that works in one language will transfer directly to another. When evaluating cross-lingual concept operations, reading accuracy, transfer performance, and erasure effectiveness need to be reported separately rather than conflated.
The study also suggests that non-compositional linguistic phenomena such as idioms may be a stricter testbed for examining cross-lingual concept sharing. Future work can extend to more language pairs and more concept categories to test whether this estimator sensitivity is universal.
Background Notes
Balanced Accuracy is a classification evaluation metric defined as the arithmetic mean of per-class recall, and it is insensitive to class imbalance. In cross-lingual concept analysis, using balanced accuracy avoids majority-class dominance and more fairly reflects a classifier's ability to recognize minority classes. In the study, it is compared against null hypotheses such as random subspaces, label permutations, and covariance matching to determine whether concept directions carry genuine cross-lingual signal. A balanced accuracy a few percentage points above baseline may be statistically significant, but the practical effect size is limited and should be interpreted with caution.
One-line Summary
A study on Bengali idioms shows that cross-lingual concept sharing is estimator-sensitive, and reading, transfer, and erasure must be evaluated separately.
Paragraph Summary
A study on Bengali idioms used six frozen encoders, 10,359 idiom items, and eight input conditions to systematically examine the reading, transfer, and erasure of cross-lingual concept subspaces. The study found that class-mean directions provide the most consistent evidence of coarse-grained sharing, outperforming multiple null hypotheses across all 12 model-label tests; English-trained class-mean readers outperformed zero-shot baselines in 8 of 12 discovery tests, and transfer depends on input mapping and regularization strength. However, reading is not erasure: English-fitted LEACE removed less than 1 percentage point of Bengali metaphor category accuracy, whereas Bengali-fitted LEACE removed 4–11 percentage points, and INLP exhibits a similar asymmetry. The study supports an estimator-sensitive interpretation of cross-lingual concept sharing and argues for separately evaluating reading, transfer, erasure, and overlap.
Related articles

Why Microsoft CEO Nadella Still Backs Nvidia After Jensen Huang's Departure
Microsoft CEO Nadella publicly expresses confidence in Nvidia interim CEO Colette Kress. This article analyzes Nvidia's power transition after Jensen Huang's departure, Microsoft's compute dependency, and the trust logic of the AI supply chain.

Nadella: Critical Infrastructure Is 'Unpatchable' as AI Cybersecurity Arms Race Escalates
Microsoft CEO Nadella addresses AI cyberattack fears, saying much critical infrastructure is unpatchable, downplaying the Outlook outage, and analyzing AI's dual role in cybersecurity.

Musk Backs Jensen Huang: Firing Him Is Like Cutting Jordan
Musk says Jensen Huang is loyal to the U.S. and likens firing him to cutting Michael Jordan—highlighting politicization anxieties in the AI chip industry and defending talent value.