New Breakthrough in Chinese Name Gender Inference: Character-Level Models Outperform Pinyin Approaches

In scientometrics research, inferring gender from names is a foundational step toward inclusive analysis. However, existing tools perform poorly on non-Western scholar names, especially Chinese names. A recent study proposes a systematic solution to this pain point, significantly improving the accuracy of Chinese name gender inference by incorporating character-level information.

The Inherent Ambiguity of Pinyin Names

Chinese names typically appear in pinyin form in international literature records, such as "Zhang Wei" or "Li Na." The problem is that the same pinyin name may correspond to multiple different Chinese character combinations, and these characters differ in their gender associations. This means that gender inference based on pinyin alone is inherently uncertain. The study points out that this ambiguity is one of the root causes of why existing methods perform poorly on Chinese names.

According to the study's analysis, names in international bibliographic databases often have only pinyin available, with Chinese character information missing. This information loss makes pinyin-based gender classifiers unable to give reliable judgments.

Design and Training of the Character-Level Models

To address the above problems, the research team first developed two character-level models: a regression model that outputs a continuous gender score from 0 (male) to 1 (female), and a classification model that outputs one of three labels: "male," "neutral," or "female." Both models were trained on rating samples from 11 native Chinese speakers who rated Chinese character names in Chinese academic publications.

On the training samples, the regression model achieved an RMSE of 0.09 and an R² of 0.85; the classification model achieved a macro-average AUC of 0.97 and a Cohen's κ of 0.81. The study emphasizes that these results remain robust under class imbalance, indicating that the models have high practical value.

RMSE (Root Mean Square Error) measures the square root of the average of the squared differences between the regression model's predicted values and the true values, with the same unit as the target variable. In this case, gender scores range from 0 to 1, and an RMSE of 0.09 means the model's predictions deviate from the true ratings by less than 0.1 on average—a high level of precision.

R² (coefficient of determination) represents the proportion of variance explained by the model, with values closer to 1 being better. An R² of 0.85 indicates that the character-level regression model can explain 85% of the variation in gender ratings.

Macro-average AUC is a commonly used evaluation metric in multi-class settings: first, AUC (area under the ROC curve, which measures ranking ability) is calculated separately for each class, and then the arithmetic mean of the per-class AUCs is taken. Macro-averaging gives equal weight to minority classes, so it reflects the model's robustness under class imbalance. An AUC of 0.97 indicates that the model is highly capable of distinguishing among the three gender categories.

Cohen's κ measures the agreement between classification results and true labels, while correcting for the probability of chance agreement. A κ of 0.81 is generally regarded as "substantial agreement," indicating that the model's labels closely match human annotations.

Aggregation Pipeline for Pinyin-Only Scenarios

For names with only pinyin available, the study proposes an aggregation pipeline: pinyin is mapped to possible Chinese characters through transliteration, and then the character-level model's predictions are aggregated using frequency weighting. At the same time, the pipeline outputs an "ambiguity score" to flag names whose gender cannot be determined from pinyin alone. This design both provides predictions and honestly exposes uncertainty, facilitating subsequent manual review or trade-offs in downstream analysis.

Transliteration here refers to mapping pinyin syllables back to the set of possible corresponding Chinese characters. Since a single pinyin syllable (such as "wei") may correspond to dozens of Chinese characters, the transliteration step generates candidate character combinations.

Frequency weighting means assigning weights to each candidate character's prediction based on its frequency of occurrence in name corpora—the more frequent the character combination, the greater its weight—thereby reducing interference from rare combinations.

The ambiguity score is an auxiliary metric output by the pipeline, used to quantify the degree of uncertainty in a pinyin name's gender attribution. For example, "Zhang Wei" may correspond to "张伟" (male-leaning) or "张薇" (female-leaning), so its ambiguity score would be high; while "Li Na" corresponds to "李娜" with overwhelming probability, so its ambiguity score would be low. This score can help downstream analyses decide whether to adopt the prediction or refer it for manual review.

Benchmark Comparison with Existing Tools

The study compared the proposed model with three off-the-shelf tools: Namsor, GPT, and Claude. The results show that the model achieved the highest agreement with human annotations in terms of both gender scores and gender labels, and for both Chinese characters and pinyin forms. This multi-tool cross-validation demonstrates that the incorporation of character-level information brings substantive improvement, rather than being merely an incidental advantage of training data or evaluation methods.

Interestingly, when applied to an independent sample of Chinese-name authors in APS journals, the model assigned a gender to every name, whereas existing pinyin-based methods left 20% to 100% of names unclassified. This comparison directly reflects the limitations of traditional methods in terms of coverage and usability.

Implications for Scientometrics

The core contribution of this work is demonstrating the necessity of character-level information for Chinese name gender inference. For researchers engaged in scientometrics, research evaluation, and academic diversity analysis, this means that shortcut methods relying on pinyin may systematically miss or misclassify a large number of Chinese names, thereby affecting the reliability of research conclusions. The research code and data have been made publicly available, providing a foundation for subsequent reproduction and extension.

From a broader perspective, this study also echoes a common problem: tools designed for Western names often fail to adapt in cross-linguistic scenarios. The solution is not simply parameter tuning, but rather returning to the characteristics of the language itself and filling in the critical information layers.

One-line Summary

A new study uses character-level models to significantly improve the accuracy of Chinese name gender inference, outperforming pinyin-based approaches.

Paragraph Summary

To address the inherent ambiguity in gender inference from pinyin Chinese names, a study developed character-level regression and classification models and built an aggregation pipeline from pinyin to Chinese characters. The regression model achieved an RMSE of 0.09 and an R² of 0.85, while the classification model achieved a macro-average AUC of 0.97. In pinyin-only scenarios, the pipeline aggregates character-level predictions through transliteration and frequency weighting, and outputs an ambiguity score. Comparisons with Namsor, GPT, and Claude show that the model has the highest agreement with human annotations. In the APS journal sample, the model classified all names, whereas pinyin-based methods missed 20% to 100% of names. The study emphasizes the necessity of character-level information for Chinese name gender inference, which is of great significance for inclusive analysis in scientometrics.