#self-attention mechanism

376 related articles

Jun 2, 2026·4 min

Core Principles of the Transformer Architecture: A Deep Dive into Self-Attention Mechanisms and Engineering Optimizations

Deep dive into Transformer architecture covering self-attention QKV mechanics, Encoder-Decoder structure, Flash Attention memory optimization, RoPE positional encoding, and GQA inference acceleration.

Jul 29, 2026·2 min

Transformer²: Achieving Co-Design of Robot Morphology and Control with a Unified Architecture

Deep dive into how Transformer² uses a unified Transformer architecture to integrate robot morphology design and motion control into one model, enabling task-driven end-to-end co-design for embodied AI.

Jul 28, 2026·3 min

The Collapse of the Lexical Assumption: Analyzing the Implicit Feature Inheritance Challenge in AI Alignment

Deep analysis of implicit feature inheritance in AI alignment: Anthropic's research reveals model behavior can propagate independently of semantics, fundamentally challenging traditional RLHF safety mechanisms.

Jul 28, 2026·2 min

From a 45-Page Plan to a 14-Page PPT: How AI Solves the Executor's Communication Dilemma

A senior executive used ChatGPT to condense a 45-page technical plan into a 14-page board PPT, revealing AI's practical value in information compression, upward communication, and workplace expression.

Jul 28, 2026·3 min

From a 45-Page Plan to a 14-Page PPT: How AI Solves the Executor's Communication Dilemma

Jul 28, 2026·3 min

Preparation Guide for NLP Research Scientist Internships: Key Focus Areas and Common Pitfalls

A comprehensive guide to preparing for NLP Research Scientist Intern roles, covering evaluation criteria, foundational knowledge, paper reading strategies, hands-on skills, and common pitfalls.

Jul 28, 2026·3 min

Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention

A deep dive into Kimi Delta Attention (KDA): tracing the evolution from quadratic Softmax attention through linear attention, Delta rules, and gated decay mechanisms, with insights on associative memory and hardware optimization.

Jul 28, 2026·3 min

Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention

Deep dive into Kimi Delta Attention (KDA): from standard Softmax attention's quadratic bottleneck through linear attention, Delta Rule, and gated decay mechanisms — the complete evolution explained.

Jul 28, 2026·2 min

Can 45 Yes/No Questions Let an LLM Identify 16 Cards? An Information Theory Perspective on AI Reasoning

Analyzing whether LLMs can identify 16 cards through 45 yes/no questions from an information theory perspective. Exploring AI reasoning capabilities in constraint-based multi-turn tasks.

Jul 28, 2026·3 min

Kimi K3 Launch: A 2.8 Trillion Parameter Flagship Model Makes Its Grand Entrance

Moonshot AI launches Kimi K3 with 2.8 trillion parameters and 1M token context. Google delays Gemini 3.5 Pro, AI coding tools upgrade collectively as competition shifts to coding and Agent capabilities.

Jul 28, 2026·3 min

Kimi K3 Open-Source Release: 2.8 Trillion Parameters Tops Global Programming Leaderboard

Moonshot AI's Kimi K3 launches with 2.8 trillion parameters, tops LMArena frontend coding leaderboard as world #1, completing tasks at one-third competitors' cost. Fully open-source for commercial use.

Jul 28, 2026·4 min

Kimi K3 Released: 2.8 Trillion Parameter Open-Source Model Tops Global Coding Leaderboard

Moonshot AI releases Kimi K3, a 2.8 trillion parameter open-source model using MoE architecture that tops the global frontend coding arena at under $1 per task, beating GPT and Claude.

Jul 28, 2026·2 min

Kimi 3 Added to Pro Subscription: Accelerating Commercialization of Chinese LLMs

Kimi 3 joins the Pro subscription plan, giving paying users direct access to the latest flagship model. Analysis of Kimi 3's upgrades, Moonshot AI's subscription strategy, and China's evolving LLM landscape.

Jul 28, 2026·2 min

Kimi 3 Included in Pro Subscription: Accelerating Commercialization of Chinese LLMs

Kimi 3 is now included in the Pro subscription plan, giving paying users direct access to the latest flagship model. This article analyzes Kimi 3's upgrades, Moonshot AI's subscription strategy, and shifts in China's LLM competitive landscape.

Jul 28, 2026·3 min

How to Elegantly Access Top AI Models Like GPT and Claude from China

Chinese users face registration, payment, and network barriers when accessing GPT, Claude, and other top AI models. This guide covers AI aggregation platforms as a one-stop solution.

Jul 28, 2026·5 min

Learn AI Agent Development from Scratch: A 3-Step Practical Path to Get Started Quickly

How to learn AI Agent development from scratch? This article outlines a clear 3-step path: Python crash course, LLM theory & practice, and LangChain framework project implementation.

Jul 28, 2026·3 min

Agent Skill Routing: Engineering Trade-offs Between Retrieval and LLM, and the Two-Stage Layered Architecture

Deep dive into Agent skill routing: comparing pure model vs. pure retrieval approaches, with a detailed two-stage layered architecture balancing accuracy, latency, and cost.

Jul 28, 2026·2 min

Kimi K3 Launches on Telnyx Inference API: A New Path for Chinese LLMs Going Global

Moonshot AI's Kimi K3 is now available on Telnyx Inference API. Explore how Chinese LLMs are entering global developer ecosystems through third-party inference platforms.

Jul 28, 2026·3 min

Three Essential Hard Skills Traditional Product Managers Need to Transition into AI PM

How can traditional product managers transition to AI PM? This article analyzes the essential differences and details three must-have skills: AI product cognition, advanced Prompt engineering, and large model technical logic.

Jul 28, 2026·3 min

AI Research Automation: More Like Data Cleaning Than Inventing the Transformer

AI research automation will look more like data cleaning than inventing the Transformer. Explore how automating 60%-80% of repetitive research work reshapes the AI research paradigm.