Unverified70% confidenceFactExact time
The pre-training corpora of current mainstream large models are highly dependent on publicly available internet data, including CommonCrawl, Wikipedia, and the GitHub codebase.
2
Sources
70%
Confidence
Long-term
Relevance
7/11/2026
First Seen
Sources
Related Claims
UnverifiedThe training data of mainstream LLMs contains a large amount of open-source embedded code, such as tens of thousands of STM32 HAL examples on GitHub.77% similarUnverified大模型在预训练阶段接触了海量GitHub上的diff格式数据，对该格式有极高的理解与生成能力75% similarUnverified现代大语言模型的预训练数据集通常包含数万亿 token，涵盖 GitHub 代码仓库、Stack Overflow、Linux 文档等75% similarVerifiedLarge language models learn the syntax rules of programming languages and the mapping between natural language and code by pre-training on billions of lines of open-source code on platforms such as GitHub.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/486679API
curl https://kongchang.com/api/v1/knowledge/claims/486679MCP
get_claim(id=486679)