Chinese Pre-trained Model Repository: An Open-Source Project with 5500+ Stars Covering LLMs and Multimodal Models

Project Overview: A One-Stop Index for Chinese Pre-trained Models

In the Chinese NLP field, finding the right pre-trained model has never been easy. Models are scattered across repositories of various organizations, versions iterate quickly, and information is highly fragmented. The awesome-pretrained-chinese-nlp-models project on GitHub (maintained by lonePatient) was created to solve this problem—it consolidates high-quality Chinese pre-trained models, large language models, and multimodal models into a single directory, systematically categorized by type and domain.

A pre-trained model refers to a neural network model that has been pre-trained on large-scale unlabeled corpora through self-supervised learning tasks (such as masked language modeling, next sentence prediction, etc.). This paradigm was pioneered by Google's release of BERT in 2018, with the core idea being "first learn general language representations on universal corpora, then fine-tune on specific tasks," dramatically reducing NLP tasks' dependence on labeled data. For Chinese, pre-trained models face unique challenges: Chinese lacks natural whitespace tokenization, requires character-vs-word granularity decisions, handles traditional/simplified character differences, and demands high-quality and diverse training corpora—all of which have led Chinese pre-trained model development to follow technical paths independent of the English-language community.

As of now, this project has earned over 5,500 Stars and 500+ Forks, making it one of the most recognized model index resources in the Chinese AI community.

Why Chinese NLP Models Need a Unified Index

The Selection Dilemma from Model Explosion

Over the past few years, the number of Chinese pre-trained models has grown extremely fast. From early Chinese versions of BERT, to large language models like ChatGLM, Qwen, DeepSeek^Ad, and Baichuan, to multimodal models supporting image-text understanding, the entire ecosystem has become quite expansive. But problems have emerged alongside this growth: these models are distributed across different platforms and different organizations' code repositories, forcing developers to search and compare one by one—a highly inefficient process.

The development of Chinese pre-trained models has gone through several distinct phases. Around 2019, encoder-only models represented by the Chinese BERT, HIT's BERT-wwm (Whole Word Masking), and RoBERTa-wwm-ext dominated Chinese NLP, excelling at text understanding tasks. ALBERT dramatically compressed model size through parameter sharing and matrix factorization, while ELECTRA replaced masked prediction with "replaced token detection" for higher training efficiency. After 2022, decoder-only models represented by the GPT architecture and encoder-decoder models (like T5) rose to prominence, giving birth to Chinese large language models including ChatGLM (Zhipu AI, based on GLM architecture), Qwen (Alibaba Cloud, based on Transformer Decoder), DeepSeek (DeepSeek AI, known for its MoE—Mixture of Experts—architecture), and Baichuan (Baichuan Intelligence). Parameter scales leaped from billions to hundreds of billions, and capabilities expanded from single-task understanding to generation, reasoning, and multi-turn dialogue.

awesome-pretrained-chinese-nlp-models serves as a "Chinese model navigation hub," pulling scattered information onto a single page, categorized by model type, applicable tasks, and vertical domains, saving significant time that would otherwise be spent on repetitive searching.

Coverage Spanning All Major Directions

This project's coverage is remarkably comprehensive, essentially spanning all major directions in the Chinese NLP ecosystem:

• Classic Pre-trained Language Models: Chinese versions of architectures like BERT, RoBERTa, ALBERT, and ELECTRA, suitable for traditional NLP tasks such as text classification and named entity recognition
• Large Language Models (LLMs): Major domestic Chinese large models including Baichuan, Qwen, Zhipu ChatGLM, and DeepSeek
• Multimodal Models: Chinese models in cross-modal directions such as vision-language models and speech-text models
• Domain-Specific Models: Industry-specific Chinese pre-trained models for healthcare, legal, finance, and other sectors

Among these, multimodal models represent one of the fastest-growing directions in recent years. Multimodal models can simultaneously process and understand multiple information modalities (such as text, images, audio, and video). Their core technical challenge lies in cross-modal alignment—enabling the model to understand the semantic correspondence between different modalities, for example, that a picture of a cat and the word "cat" refer to the same concept. Typical technical approaches include CLIP-style contrastive learning (mapping images and text into a shared vector space), Flamingo-style vision-language cross-attention mechanisms, and the recently popular approach of bridging visual encoders (such as ViT) with large language models through projection layers. In the Chinese domain, representative multimodal models include Alibaba's Qwen-VL and Zhipu's CogVLM/GLM-4V, which can perform Chinese image-text QA, image captioning, document OCR understanding, and other tasks.

Domain-specific models represent another important trend. These are specialized models obtained by performing continual pre-training or domain-adaptive fine-tuning on top of general pre-trained models using industry-specific corpora. Taking the medical domain as an example, general models may not accurately understand the semantic relationships of professional terms like "atrial fibrillation" or "ejection fraction," but after continued training on large volumes of medical literature, clinical records, and drug instructions, models can significantly improve their medical text comprehension capabilities. Similarly, legal domain models need to understand special language patterns like statute citations and case-based reasoning, while financial domain models need to master specialized knowledge in financial report analysis and risk assessment. The emergence of these vertical models reflects the industry's pragmatic shift from "one general-purpose large model solves everything" to "general + specialized collaboration."

Core Value of the Project: More Than Just a Link Collection

Structured Model Information

Unlike ordinary link lists, this project provides structured descriptions for each model, including key information such as the source organization, parameter scale, training data overview, and open-source license. Developers can quickly determine whether a model meets their needs without having to open each project's homepage individually.

This structured presentation is particularly useful for scenarios requiring side-by-side comparison of multiple Chinese pre-trained models.

Community-Driven, Continuously Updated

The Chinese large model field iterates at a rapid pace, with new models released almost every month. This project maintains an active update cadence, with new models being promptly included. Additionally, community contributors can submit newly discovered models via Pull Requests, creating a collaborative maintenance mechanism that ensures information timeliness.

Reducing Trial-and-Error Costs in Technical Selection

For enterprises and research teams, the choice of base model often directly impacts project effectiveness and cost. The panoramic Chinese model directory provided by this project allows technical decision-makers to quickly grasp what options are currently available, compare them across dimensions such as parameter scale, open-source license, and domain adaptation, and reduce detours during the selection phase.

Efficient Usage Guide: Three Steps to Find the Right Model

Step 1: Locate the Category Based on Task Requirements

If your needs are clear—for example, text classification, named entity recognition, dialogue generation, or document summarization—you can jump directly to the corresponding model category to see which Chinese pre-trained models are available.

Step 2: Filter by Parameter Scale

Computing constraints in the deployment environment are an unavoidable reality. The project annotates each model's parameter scale, allowing you to quickly exclude models that exceed your hardware capacity. For example, deploying on edge devices might require focusing on lightweight Chinese models under 1B parameters; if you have GPU cluster support, you can consider larger-scale LLMs.

Understanding the correspondence between parameter scale and deployment compute is a prerequisite for making sound selections. Taking FP16 (half-precision floating point) inference as an example, a 1B (1 billion) parameter model requires approximately 2GB of VRAM, a 7B model needs about 14GB, and a 70B model requires approximately 140GB of VRAM—meaning a single consumer-grade GPU (such as the RTX 4090 with 24GB VRAM) can barely run a 7B-level model, while 70B models typically require clusters of multiple A100 (80GB) or H100 GPUs. In recent years, quantization techniques (such as 4-bit/8-bit quantization in GPTQ, AWQ, GGUF formats) have become key methods for lowering deployment barriers, compressing VRAM requirements to 1/4 to 1/2 of the original, though at the cost of some accuracy loss. For edge device and mobile scenarios, lightweight models under 1B (such as MiniCPM, Qwen-0.5B, etc.) combined with INT4 quantization can even run on mobile phone chips.

Step 3: Verify the Open-Source License

This step is easily overlooked but critically important. The open-source licenses of different Chinese pre-trained models vary significantly: some are restricted to academic research only, some allow commercial use but require authorization applications, and others adopt permissive licenses like Apache 2.0. Before making a final selection, always confirm the license terms to avoid downstream compliance risks.

Specifically, common license types include: Apache 2.0—the most permissive, allowing commercial use, modification, and distribution, requiring only copyright notice retention; MIT License—equally permissive with even fewer restrictions; GPL family—requiring derivative works to also be open-sourced, unfriendly to commercial closed-source products. Additionally, many Chinese large models adopt custom licenses; for example, some models allow free commercial use but set user scale thresholds (such as requiring separate applications when monthly active users exceed a certain number), while others distinguish between "free for academic research, paid for commercial use." Recently, community licenses like those of the Llama series have emerged, allowing commercial use but prohibiting use for training competing models. In actual projects, especially those involving ToB delivery or product launches, license compliance review should be completed before technical evaluation.

Long-term Significance for the Chinese AI Ecosystem

The value of this project extends beyond information aggregation. From initially cataloging a few dozen models to now encompassing a vast directory of hundreds of Chinese pre-trained models, it is itself a microcosm of Chinese NLP development history—from the early days of catching up with the English-language community to now running alongside and even partially leading international frontiers in large language models and multimodal models.

For developers new to Chinese NLP, this project serves as a practical learning map that helps you quickly build an understanding of the entire model ecosystem; for seasoned practitioners, it's an efficient model discovery tool that eliminates the hassle of repetitive searching.

Regardless of what stage you're at, it's recommended to bookmark this repository as a go-to reference for your daily Chinese NLP work.

Project Link: GitHub - lonePatient/awesome-pretrained-chinese-nlp-models