Feed Entities Podcasts Deep Dives Developers About

[KongchangAI]

Real-time AI knowledge base. Every claim is traceable to its source, with data freshness under 24 hours.

Powered by 240+ first-hand AI sources

Knowledge

Feed
Hot Entities
Podcasts

Developers

CLI
MCP Server
REST API
About

© 2026 KongchangAI. The freshest, most accurate AI knowledge

#Tiny-MoE

1 related articles

15-Year-Old Trains 200M MoE Language Model from Scratch: Full Architecture & Hands-On Breakdown

Jul 13, 2026·2 min

15-Year-Old Trains 200M MoE Language Model from Scratch: Full Architecture & Hands-On Breakdown

A 15-year-old trained Tiny-MoE, a 200M-parameter MoE language model from scratch using free Kaggle GPUs, featuring MLA attention, RoPE+YaRN, and native PyTorch.