#RoPE positional encoding
117 related articles
·3 min
Training a 1.3B Parameter LLM from Scratch: Complete Workflow and Core Challenges
A deep dive into the complete workflow of training a 1.3B parameter LLM from scratch, covering Transformer architecture design, data preparation, and distributed training optimization.
Read more →
·4 min
SELENE Open-Source Project: An Interactive Notebook Library for Systematically Learning AI and Machine Learning from Scratch
SELENE is an open-source AI learning resource built on Jupyter Notebooks, systematically covering ML, deep learning, Transformers, and LLMs with interactive code and math derivations for beginners.
Read more →
·3 min
TokenTown: A Visual Approach to Understanding How LLMs Predict the Next Token
TokenTown is an open-source visualization project that intuitively presents the internal token prediction process of LLMs using a town metaphor. Learn its design philosophy and educational value.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.
Read more →
·3 min
Kimi K3 Reaches 2.5T Parameters, MiniMax M3 Pro Brewing Simultaneously: China's LLM Arms Race Escalates
Kimi K3 adopts new architecture with 2.5T parameters and 1M token context. MiniMax M3 Pro targets 2.7-3T. OpenAI surpasses 7M users, HY-OCR 1.5 achieves SOTA, Amap launches world model.
Read more →
·3 min
Kimi K3 Hands-On Review: 2.5 Trillion Parameter Open-Source Model Challenges Claude
Kimi K3 hands-on review: Moonshot AI's 2.5T parameter MoE model matches Claude in coding, surpasses it in 3D game development, with API pricing at one-tenth the cost of competitors.
Read more →
·3 min
Kimi K3 Hands-On Review: 2.5 Trillion Parameter Open-Source Model Takes on Claude
Hands-on review of Kimi K3, Moonshot AI's latest 2.5T parameter MoE model. Coding ability ties with Claude, surpasses it in 3D game dev, with API pricing at one-tenth of competitors.
Read more →
·3 min
Kimi K3 Launch: A 2.8 Trillion Parameter Flagship Model Makes Its Grand Entrance
Moonshot AI launches Kimi K3 with 2.8 trillion parameters and 1M token context. Google delays Gemini 3.5 Pro, AI coding tools upgrade collectively as competition shifts to coding and Agent capabilities.
Read more →
·3 min
Kimi K3 Open-Source Release: 2.8 Trillion Parameters Tops Global Programming Leaderboard
Moonshot AI's Kimi K3 launches with 2.8 trillion parameters, tops LMArena frontend coding leaderboard as world #1, completing tasks at one-third competitors' cost. Fully open-source for commercial use.
Read more →
·4 min
Kimi K3 Released: 2.8 Trillion Parameter Open-Source Model Tops Global Coding Leaderboard
Moonshot AI releases Kimi K3, a 2.8 trillion parameter open-source model using MoE architecture that tops the global frontend coding arena at under $1 per task, beating GPT and Claude.
Read more →
·5 min
Learn AI Agent Development from Scratch: A 3-Step Practical Path to Get Started Quickly
How to learn AI Agent development from scratch? This article outlines a clear 3-step path: Python crash course, LLM theory & practice, and LangChain framework project implementation.
Read more →
·3 min
Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide
Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.
Read more →
·3 min
Deep Dive into the Kimi-K3 Technical Report: A Comprehensive Analysis of Architecture, Long-Context Processing, and Reasoning Capabilities
Deep dive into Moonshot AI's Kimi-K3 technical report, analyzing its long-context processing, MoE architecture, reasoning improvements, and its position in global AI competition.
Read more →
·3 min
Deep Dive into the Kimi-K3 Technical Report: Architecture, Long Context, and Reasoning Capabilities
Deep analysis of Moonshot AI's Kimi-K3 technical report covering long context processing, MoE architecture, reasoning capabilities, and China's position in the global AI competition.
Read more →
·4 min
The More Complex the AI Agent, the More It Hallucinates? The Hidden Cost of Architectural Complexity
Why do AI Agents hallucinate more as they grow more complex? This article analyzes the causes from error accumulation, context noise, and model completion nature, with 5 practical production strategies.
Read more →
·2 min
Kimi-K3 Lands on HuggingFace: A Full Breakdown of Moonshot AI's Open-Source Strategy
Moonshot AI's Kimi-K3 model is set to launch on HuggingFace. A deep dive into its architecture, licensing, community reactions, and strategic implications for the global open-source AI landscape.
Read more →
·7 min
Getting Started with Claude Code: Five Dimensions to Understand Why It Outperforms Regular AI
Claude Code isn't just a chat AI—it can directly read projects, modify code, and run commands. This article compares Claude Code with regular AI across five dimensions to help you decide if it's worth trying.
Read more →
·7 min
DeepSeek's Mysteriously Retracted Paper: Breaking AI's Multimodal Reasoning Bottleneck with Visual Primitives
DeepSeek's paper 'Thinking with Visual Primitives' was online for just 4 hours before being pulled. It uses bounding boxes and points as reasoning primitives, letting models 'point at' images to outperform GPT, Gemini, and Claude on maze navigation and counting.
Read more →
·7 min
Context Engineering: A Systematic Approach to Stop AI Agents from 'Talking Nonsense'
Context engineering is the core methodology for building efficient AI Agents, covering query enhancement, RAG retrieval, prompt design, memory management, and tool invocation. Master Write, Select, Compress, and Isolate to solve LLM hallucination at its root.
Read more →