#GPU memory hierarchy
48 related articles
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.
Read more →
·3 min
Kimi K3 vs Claude Real-World Test: Near-Equal Performance at 1/8 the Price
Real-world comparison of Kimi K3 vs Claude flagship across e-commerce pages, 3D fighting games, and flight simulators. Kimi K3 delivers 90% output quality at 1/8 the price with faster speeds and local deployment support.
Read more →
·4 min
Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop
Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.
Read more →
·3 min
Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide
Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.
Read more →
·4 min
The Dao of DeepSeek: Liang Wenfeng on Vision, Restraint, and the Philosophy of Open Source
Analysis of DeepSeek founder Liang Wenfeng's rare investor dialogue, exploring the company's vision-driven culture, strategic restraint toward AGI, and open-source philosophy in the US-China AI race.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment
A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics on Model Quantization and Inference Deployment
A systematic guide to must-know AI application engineer interview topics: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·6 min
DiffusionGemma Character Motion Transfer in Practice: A ComfyUI Workflow Breakdown
Explore a character motion transfer experiment based on a DiffusionGemma custom node—swap identity in ComfyUI using just a static image, a reference video, and one prompt. A breakdown of the tech stack, control signal preservation, and real limitations for AI video creators.
Read more →
·6 min
Hands-On with Panel AI On-Premises Deployment: One-Click Enterprise Adoption and Unified Compute Cluster Scheduling
In-depth review of Panel AI v1.1.1: second-level installation, no-public-IP networking, batch compute cluster management. Learn how enterprise AI on-premises deployment barriers are dramatically lowered.
Read more →
·7 min
Qwen3 235B A22B in Practice: A 2.4-Trillion-Parameter Open Model Surpasses Claude Opus
Hands-on with Alibaba Tongyi Qianwen's strongest Qwen3: a 2.4-trillion-parameter open weight model scoring 81.25% on KingBench, ranking second and beating Claude Opus 4.8 with perfect scores in game dev, math, and agent tasks.
Read more →
Handwritten C/CUDA Inference Engine: P…
·2 min
Handwritten C/CUDA Inference Engine: Pushing Qwen 35B to the Limits on RTX 5090
A deep dive into a handwritten C/CUDA inference engine for Qwen 35B on RTX 5090 (Blackwell), covering quantization, FlashAttention kernels, and memory optimization.
Read more →
·3 min
NVIDIA's Dual-Tower Diffusion Language Model Explained: How Freezing Half the Network Achieves 2.4× Speedup
NVIDIA's dual-tower diffusion LM freezes the context tower while fully retraining the denoiser, achieving 2.4× speedup with 98.7% quality retention. A deep dive into the architecture, performance, and fragility.
Read more →
AirLLM: How a 4GB GPU Can Run a 70B Mo…
·2 min
AirLLM: How a 4GB GPU Can Run a 70B Model — Core Principles and Use Cases
AirLLM is an open-source project that uses layer-by-layer inference to run 70B LLMs on a single 4GB GPU. Learn how it works, its tradeoffs, and ideal use cases.
Read more →
High-Bandwidth Flash (HBF): A New Path…
·2 min
High-Bandwidth Flash (HBF): A New Path to Breaking the AI Storage Bottleneck
High-Bandwidth Flash (HBF) bridges the gap between HBM and NAND, offering high-bandwidth weight storage at lower cost to tackle the memory wall bottleneck in large AI model inference.
Read more →
·5 min
A Deep Dive into Vector Search for AI Agents: The Technical Foundations of RAG and Long-Term Memory
An in-depth look at the core tech behind AI Agents: how the HNSW, IVF, and PQ vector search algorithms power RAG and long-term memory. Understand where a model's "memory" and "knowledge" come from.
Read more →
·2 min
Training a CPU-Native LLM on Zero Budget: The Architectural Gamble of Ternary Quantization × Sparse MoE
An indie dev attempts to train a CPU-native LLM on $0 budget using ternary quantization, sparsity, and fine-grained MoE — with pre-registered success criteria and full public reporting.
Read more →
·6 min
A Deep Dive into OpenAI Codex: How AI Engineers Are Reimagining Software Development with Agents
OpenAI Codex is redefining how AI engineers work: from code completion to autonomous Agents, from single-threaded to parallel Value Maxing. A deep dive into the Codex App architecture, open ecosystem, and Manager of Agents practice.
Read more →
CUDA Kernel Fusion: A Practical Guide …
·2 min
CUDA Kernel Fusion: A Practical Guide to Reducing Memory Bandwidth and Launch Overhead
Learn how CUDA Kernel Fusion merges multiple GPU kernels to reduce global memory traffic and launch overhead, with real-world examples from AI inference and deep learning.
Read more →
Taiwan's Lost 8-Bit Computers: A Forgo…
·5 min
Taiwan's Lost 8-Bit Computers: A Forgotten Chapter of Asian Tech History
During the 8-bit era, Taiwan produced homegrown computers now nearly forgotten. This article traces that little-known history and its impact on Taiwan's tech rise.
Read more →