#KDA attention
31 related articles
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.
Read more →
·3 min
Comparing China's Four Major AI Labs: Strategic Analysis of Qwen, DeepSeek, Moonshot, and Ling
An insider's analysis of China's four AI labs — Qwen, DeepSeek, Moonshot, and Ling — revealing their distinct strategic bets on distribution, architecture, long-termism, and serving cost.
Read more →
·3 min
How to Find High-Paying Remote AI/ML Jobs? A Complete Practical Job Search Strategy Guide
In-depth analysis of job search strategies for high-paying remote AI/ML and data analytics roles, covering referrals, niche communities, personal branding, and salary negotiation tactics.
Read more →
·3 min
Running the 1.56TB Kimi K3 Model on 8GB RAM with Pure C
A developer built a pure C99 inference engine that runs the 1.56TB Kimi K3 model on 8GB RAM using MoE sparsity and NVMe on-demand loading—no GPU, 176KB binary.
Read more →
·5 min
witr: A Linux Process Tracing Tool for Quickly Identifying Why a Process Is Running
witr (Why Is This Running) is an open-source Linux process tracing tool that traces startup chains from processes, ports, containers, or files to quickly identify whether a process was launched by systemd, cron, or supervisor.
Read more →
·3 min
GitHub Daily · July 29: Voice AI and On-Device Training Take Flight as Open-Source Alternatives Rise
GitHub Trending July 29: Microsoft's VibeVoice leads voice AI open-source wave, MoonshotAI's FlashKDA CUDA kernel surges 25%, and open-source alternatives rise.
Read more →
·3 min
Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention
A deep dive into Kimi Delta Attention (KDA): tracing the evolution from quadratic Softmax attention through linear attention, Delta rules, and gated decay mechanisms, with insights on associative memory and hardware optimization.
Read more →
·3 min
Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention
Deep dive into Kimi Delta Attention (KDA): from standard Softmax attention's quadratic bottleneck through linear attention, Delta Rule, and gated decay mechanisms — the complete evolution explained.
Read more →
·3 min
The Real Daily Life of MLE and MLOps Engineers: Key Differences Between Three Roles and Career Paths
An in-depth look at the real daily work of data scientists, MLEs, and MLOps engineers — covering responsibilities, essential tools, and career paths to help you find your direction in AI.
Read more →
·2 min
DeepSeek V4 Official Release Imminent: What 1.6 Trillion Parameters Really Means
DeepSeek V4 official release is imminent. V4 Pro features 1.6 trillion parameters with million-character context. Explore the D-SPARK acceleration framework, dynamic pricing, and V4's leap to production-grade AI.
Read more →
·5 min
From Chat to Agent: Automating Your Entire Business Workflow with AI Agents
Veteran AI practitioner Remy breaks down the leap from chat models to AI agents: how agents work, the three pillars of context, tools, and skills, MCP connections, and hands-on architecture to make you a 100x employee.
Read more →
·3 min
Kimi K3 Released: How a 2.8 Trillion Parameter Open Model Reshapes AI Cost-Effectiveness
Moonshot AI unveils Kimi K3: a 2.8 trillion parameter, 1M context, natively multimodal open model. With KDA architecture and ultra-low cost, it rivals GPT-5.6 and Fable 5, redefining AI cost-effectiveness.
Read more →
·3 min
Is Gemini Getting Worse? A Deep Dive into the 'Model Degradation' Phenomenon
Reddit users report Gemini feeling "worse over time." This deep dive explains the real causes—silent updates, safety over-alignment, and compute throttling—plus how to rationally evaluate AI model performance.
Read more →
·2 min
17 Million Mouse Trajectories: Unpacking the Machine Learning Value of Gaming Behavioral Data
A League of Legends player collected 17M mouse trajectories and 670K clicks. We analyze the ML value of this gaming behavioral telemetry data for imitation learning, anti-cheat, and player modeling.
Read more →
Fine-Tuning Cosmos Models in One Day w…
·2 min
Fine-Tuning Cosmos Models in One Day with Agent Skills — Breaking 90% Accuracy
NVIDIA uses Autonomous Coding Agents and Agent Skills with TAO to fine-tune Cosmos visual reasoning models in one day, achieving over 90% accuracy.
Read more →
·3 min
Sent Hundreds of Resumes with No Response? A Deep Dive into Tech Job Search Struggles and How to Break Through
Sending hundreds of resumes with no response? This article breaks down the core pain points in today's tech job market — ATS filtering, spray-and-pray fatigue, neglected referral channels — and offers actionable strategies to break through.
Read more →
·3 min
Tracking London's Underground in 3D Real-Time: A Technical Breakdown of an Open-Source Visualization Project
An open-source project rendering the London Underground's real-time status in 3D, combining the TfL open API, WebGL rendering, and real-time data streaming. A deep dive into its implementation and interpolation challenges.
Read more →
·4 min
4 Skills to Completely Transform Claude Code: Say Goodbye to Token Waste and Cut Daily Coding to 2 Hours
Does Claude Code always give irrelevant answers? This article breaks down 4 core Skill plugins: Project Context Engine, PRD Requirements Translator, Code Review tool, and Daily Report Generator—showing you how to transform Claude Code from a money-burner into a true AI coding assistant.
Read more →
·2 min
AI Agents for SGLang Kernel Optimization: Real-World Performance Gains and Engineering Methodology
SGLang's team converted expert knowledge into agent skills, achieving 71.4% throughput gains, TTFT reduced from 456ms to 168ms. A deep dive into agent-assisted kernel optimization methodology.
Read more →