#kernel launch overhead
116 related articles
·6 min
AI Application Engineer Interview Guide: Core Topics on Model Quantization and Inference Deployment
A systematic guide to must-know AI application engineer interview topics: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment
A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·7 min
Getting Started with Codex: Installation and Usage Guide for OpenAI's AI Coding Agent
A comprehensive guide to OpenAI's new AI coding agent Codex: from concept and comparison of its four forms, to installing Git/Node.js/VS Code, configuring the API Key, and creating a workspace.
Read more →
·7 min
The Complete Guide to LLM Application Engineer Interviews: Six High-Frequency Agent Topics
A focused guide to core LLM application engineer interview topics, covering agent architecture, Multi-Agent, Langfuse evaluation, security, and RAG optimization.
Read more →
·7 min
Alibaba Open-Sources 2.4T Qwen Model as Chinese AI Sweeps the Global Competitive Landscape
Alibaba open-sources a 2.4 trillion parameter Qwen model and launches the Qwen Token Plan. Chinese models surge, Kimi K3 tops global rankings, and China's AI is reshaping the global competitive landscape.
Read more →
Handwritten C/CUDA Inference Engine: P…
·2 min
Handwritten C/CUDA Inference Engine: Pushing Qwen 35B to the Limits on RTX 5090
A deep dive into a handwritten C/CUDA inference engine for Qwen 35B on RTX 5090 (Blackwell), covering quantization, FlashAttention kernels, and memory optimization.
Read more →
Why We Must Actively Fund Open Source …
·2 min
Why We Must Actively Fund Open Source AI: A Public Goods Perspective on Technology Democratization
Open source AI faces soaring compute costs and fierce talent competition that markets alone can't solve. This deep dive explores why actively funding open source AI is essential for tech democratization.
Read more →
Java Local LLM Inference: Low-Latency …
·2 min
Java Local LLM Inference: Low-Latency LLM Deployment with Panama FFM
Learn how Java and OpenJDK Panama FFM API enable local LLM inference. Explore the technical foundations, JVM ecosystem benefits, and low-latency AI deployment in enterprise Java systems.
Read more →
·3 min
GPUStack in Action: Complete Guide to Day 0 Deployment of DeepSeek-V4-Flash
Deploy DeepSeek-V4-Flash DSpark on 8× H20-141G using GPUStack's SGLang backend on Day 0. Full walkthrough of Web UI config, parameter tuning, and 200 tokens/sec benchmark results.
Read more →
·2 min
ExLlamaV3 v1.0.0 Official Release: Faster Inference, Lossless KV Quantization, and Streamlined Dependencies
ExLlamaV3 v1.0.0 releases with lossless KV cache quantization via kernel fusion, removal of flash-attention-2/xformers, major GEMM/GEMV gains, and broader tensor parallelism support.
Read more →
·3 min
Getting Started with Dify: A Complete Guide to Building Enterprise-Grade AI Apps with Low-Code
A complete guide to Dify, the low-code AI app platform: five app types, multi-model setup, Docker deployment, and enterprise data security. Build LLM-powered workflows and Agents at minimal cost.
Read more →
4 Alternatives for Running CUDA on Non…
·3 min
4 Alternatives for Running CUDA on Non-NVIDIA Hardware: A Comparative Guide
A deep dive into running CUDA on non-NVIDIA hardware (AMD, Intel): comparing ROCm/HIP, ZLUDA, SYCL/oneAPI, and OpenCL across principles, use cases, and limitations.
Read more →
Zig Creator Calls Out Anthropic: The G…
·2 min
Zig Creator Calls Out Anthropic: The Gap Between AI Hype and Technical Reality
Zig creator Andrew Kelley publicly criticizes Anthropic for "blowing smoke" in AI marketing. A deep dive into the tension between AI hype and engineering integrity.
Read more →
·4 min
Home Assistant: The Local-First Open Source Smart Home Platform That Puts Privacy and Freedom First
Home Assistant is the most mature open source smart home hub, prioritizing local control and privacy, supporting thousands of devices across ecosystems with 88,000+ GitHub stars.
Read more →
·7 min
Breaking Down the Core Skills of an AI Application Engineer: From Beginner to Industrial-Grade Deployment
Knowing how to call an API doesn't make you an AI engineer. This article breaks down the complete skill structure of an AI application engineer, covering Python fundamentals, LLM fine-tuning, Agent development, and enterprise projects.
Read more →
·2 min
SGLang Integrates DSpark: How Confidence-Driven Speculative Decoding Cracks the High-Concurrency Failure Problem
SGLang officially integrates DSpark, solving the core pain point of speculative decoding failure under high-concurrency batches via confidence-driven variable-length verification. Supports Qwen3 and DeepSeek-V4, hitting 383.7 tok/s on B300.
Read more →
·6 min
DeepSeek Builds Its Own AI Chips: The Strategic Logic and Challenges Behind Compute Independence
DeepSeek is entering AI chip development, targeting compute autonomy. This article analyzes its motivations, software-hardware synergy, chip R&D challenges, and impact on China's AI vertical integration.
Read more →
·4 min
Deep Dive into Asio: The De Facto Standard and Core Design of C++ Asynchronous Network Programming
A comprehensive analysis of Asio (Boost.Asio) covering its Proactor async model, completion token mechanism, C++20 coroutine support, and cross-platform I/O abstraction for high-performance C++ networking.
Read more →
·7 min
Practical RL Compute Optimization: Core Strategies to Boost CPU and GPU Utilization
An in-depth look at why CPU and GPU utilization is low in RL training, covering vectorized environment parallelism, distributed Actor-Learner architectures, GPU-side simulation (Isaac Gym/Brax), and Ray RLlib practice.
Read more →
·4 min
Dify Getting Started: A Complete Guide to Building AI Workflows Without Code
A complete guide to Dify's core features and 1.8.0 deployment. Covers 5 app types, Docker setup, Workflow vs Chatflow differences, and RAG knowledge bases for beginners.
Read more →