Unverified50% confidenceReleaseExact time
Tencent U2LLM A7B Base is open-sourced, a vision-language MoE pretrained model with about 120 billion total parameters, about 7 billion activated per token, a training context of 262,144 tokens, and training on 11.5 trillion multimodal tokens; architecturally it uses hybrid attention interleaving KDA and Gated MLA, with the MoE part having 256 routed experts, Top8 plus one shared expert, 48 layers in total, and a Qwen 3 VL MoE vision encoder; in evaluations, MMLU Pro 5-Shot EM is 78.61, GPQA Main is 68.53, and MBPP+ is 89.15.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/25/2026
First Seen
Valid until: 1/23/2027
Sources
Related Entities
Related Claims
UnverifiedTiny-MoE在80亿(8B)token上从零训练,使用FineWeb-Edu、Cosmopedia v2和OpenWebMath数据集64% similarUnverifiedThe experimental setup used a single NVIDIA H100 serving via vLLM, with test models Gemma 4 12B and Gemma 4 31B, and input data consisting of 50 million tokens of real public text.62% similarUnverifiedThe training of modern large language models relies at the computing level on specialized GPU chips from companies such as NVIDIA and cloud service providers such as AWS and Microsoft Azure.61% similarUnverified向量嵌入通常使用针对代码训练的嵌入模型(如OpenAI的text-embedding-3、Voyage AI的voyage-code-2),将代码块映射为768维或1536维向量60% similarUnverifiedOpenBMB发布了面向LLM预训练的开放英文网页语料库UltraFineWeb,总量超过1万亿token、约11.4亿篇文档,全部来自Common Crawl并以Apache 2.0许可证开放60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1094870API
curl https://kongchang.com/api/v1/knowledge/claims/1094870MCP
get_claim(id=1094870)