#multi-head attention
148 related articles
·2 min
GPT-5.6 Price Cut of 80%: How OpenAI Reclaims the AI Price-Performance Throne
OpenAI releases GPT-5.6 with 80% price cuts on Luna models, overtaking DeepSeek on price-performance. Analysis of the tech logic, developer impact, and AI pricing trends.
Read more →
·2 min
Why Top AI Startups Are No Longer Publishing Research: The Deep Reasons Behind the Shift from Openness to Secrecy
Exploring why top AI startups shifted from open research to secrecy, analyzing how commercial competition and talent pressure drive this change, and its impact on academia, innovation, and open source.
Read more →
·3 min
TokenTown: A Visual Approach to Understanding How LLMs Predict the Next Token
TokenTown is an open-source visualization project that intuitively presents the internal token prediction process of LLMs using a town metaphor. Learn its design philosophy and educational value.
Read more →
·3 min
Preparation Guide for NLP Research Scientist Internships: Key Focus Areas and Common Pitfalls
A comprehensive guide to preparing for NLP Research Scientist Intern roles, covering evaluation criteria, foundational knowledge, paper reading strategies, hands-on skills, and common pitfalls.
Read more →
·5 min
Learn AI Agent Development from Scratch: A 3-Step Practical Path to Get Started Quickly
How to learn AI Agent development from scratch? This article outlines a clear 3-step path: Python crash course, LLM theory & practice, and LangChain framework project implementation.
Read more →
·4 min
Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop
Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.
Read more →
·4 min
Kimi K3 Deep Dive: 2.8 Trillion Parameter Open-Source Model Closes the Gap with Closed-Source Frontier for the First Time
Moonshot AI releases Kimi K3 open-weight model with 2.8T parameters and 1M token context. Our deep dive covers coding, 3D dev, agent capabilities, and safety concerns.
Read more →
·3 min
AI Research Automation: More Like Data Cleaning Than Inventing the Transformer
AI research automation will look more like data cleaning than inventing the Transformer. Explore how automating 60%-80% of repetitive research work reshapes the AI research paradigm.
Read more →
·3 min
AI Research Automation: More Like Data Cleaning Than Inventing the Transformer
Why AI research automation looks more like data cleaning than inventing the Transformer. Exploring the value of automating 60%-80% of repetitive research work and how human-AI collaboration reshapes the research paradigm.
Read more →
·3 min
Kimi K3 Launch: Performance Rivaling Claude at One-Third the Price, US-China AI Gap Narrows to 3 Months
Moonshot AI launches Kimi K3 reasoning model with performance rivaling Claude and OpenAI's top models at one-third the price. The US-China AI gap narrows from 6-12 months to just 3 months.
Read more →
·4 min
The Dao of DeepSeek: Liang Wenfeng on Vision, Restraint, and the Philosophy of Open Source
Analysis of DeepSeek founder Liang Wenfeng's rare investor dialogue, exploring the company's vision-driven culture, strategic restraint toward AGI, and open-source philosophy in the US-China AI race.
Read more →
·4 min
Chinese AI Accounts for 58% of US Compute Consumption: How Open-Source LLMs Are Disrupting Silicon Valley
OpenRouter data shows Chinese AI models now account for 58% of US AI consumption. Silicon Valley giants like DoorDash and Airbnb adopt Kimi, DeepSeek, and Qwen for their low cost and open-weight advantages.
Read more →
·4 min
Chinese AI Accounts for 58% of US Compute Consumption: How Open-Source LLMs Are Disrupting Silicon Valley
OpenRouter data shows Chinese AI models now account for 58% of US AI consumption. Silicon Valley giants like DoorDash and Airbnb adopt Kimi, DeepSeek, and Qwen, leveraging low cost, high performance, and open weights.
Read more →
·2 min
Jensen Huang Praises DeepSeek and Kimi: Markets Have Twice Misjudged the Impact of Chinese Open-Source Models
NVIDIA CEO Jensen Huang says markets have twice misjudged the impact of DeepSeek and Kimi, arguing Chinese open-source models boost rather than reduce overall AI compute demand.
Read more →
·3 min
Deep Dive into the Kimi-K3 Technical Report: A Comprehensive Analysis of Architecture, Long-Context Processing, and Reasoning Capabilities
Deep dive into Moonshot AI's Kimi-K3 technical report, analyzing its long-context processing, MoE architecture, reasoning improvements, and its position in global AI competition.
Read more →
·3 min
Deep Dive into the Kimi-K3 Technical Report: Architecture, Long Context, and Reasoning Capabilities
Deep analysis of Moonshot AI's Kimi-K3 technical report covering long context processing, MoE architecture, reasoning capabilities, and China's position in the global AI competition.
Read more →
·2 min
Kimi-K3 Lands on HuggingFace: A Full Breakdown of Moonshot AI's Open-Source Strategy
Moonshot AI's Kimi-K3 model is set to launch on HuggingFace. A deep dive into its architecture, licensing, community reactions, and strategic implications for the global open-source AI landscape.
Read more →
·3 min
Kimi K3 Local Deployment Guide: Storage Requirements for q4 and q8 Quantization Explained
Detailed analysis of Kimi K3 quantization deployment options, comparing q4 vs q8 storage requirements, precision trade-offs, and hardware configurations for local self-hosting.
Read more →
·4 min
FPGA Softmax Approximation in Practice: A Detailed Guide to Taylor Expansion and Padé Approximation Methods
A detailed guide on efficiently implementing Softmax on FPGAs, comparing Taylor series and Padé approximation methods for accuracy and resource trade-offs, with Python simulation and range reduction techniques.
Read more →
Dive into LLMs: A Complete Guide to th…
·1 min
Dive into LLMs: A Complete Guide to the 44.8K-Star Hands-On LLM Tutorial
"Dive into LLMs" is a 44,830-star Chinese LLM tutorial on GitHub. Using Jupyter Notebooks, it covers Transformers, LoRA fine-tuning, RAG, and Prompt Engineering.
Read more →