#GPU memory bandwidth

332 related articles

NVIDIA GQE Deep Dive: How GPU Query En…

Jul 7, 2026·2 min

NVIDIA GQE Deep Dive: How GPU Query Engines Break Through Memory Bandwidth Bottlenecks

A deep dive into NVIDIA GQE's architecture: how HBM, NVLink, and memory-hierarchy-aware execution models help GPU query engines overcome I/O and bandwidth bottlenecks.

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Why memory bandwidth (GB/s), not VRAM size, determines local LLM inference speed. Includes tokens/sec formula, GPU bandwidth comparison, and a practical card selection framework.

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Learn why memory bandwidth (GB/s)—not VRAM size—determines local LLM inference speed. Get the tokens/sec formula, GPU bandwidth comparisons, and a practical card selection hierarchy.

Jul 30, 2026·2 min

SDL_GPU: Deep Dive into a Single-Header High-Performance 2D Graphics Library

Deep analysis of the SDL_GPU single-header 2D graphics library covering its design philosophy, GPU acceleration principles, use cases, and technical trade-offs.

Jul 30, 2026·2 min

Speech To Markdown: Local AI Voice-to-Structured Markdown Note-Taking Tool

Speech To Markdown is a free macOS/iOS app that converts voice to structured Markdown notes using local LLMs. Fully offline, no API keys needed, with global hotkey dictation.

Jul 30, 2026·3 min

Is LLM Inference Really That Expensive? Breaking Down the Cost Reduction Logic of GPU Compute and Inference Frameworks

Analyzing real LLM inference costs: from B200 GPU compute gains, vLLM framework optimization to MTP multi-token prediction, explaining why serving costs are widely overestimated.

Jul 30, 2026·3 min

GPT-5.6 Autonomously Optimizes Its Own Kernels: How AI Recursive Self-Improvement Could Cut Service Costs by 20%

OpenAI's internal model GPT-5.6 reportedly autonomously rewrites production kernels, achieving ~20% service cost reduction. Deep analysis of this AI recursive self-optimization event's technical plausibility and industry impact.

Jul 30, 2026·3 min

GPT-5.6 Autonomously Optimizes Its Own Kernels: How AI Recursive Self-Improvement Could Cut Service Costs by 20%

OpenAI's internal model GPT-5.6 reportedly autonomously rewrote production compute kernels, achieving ~20% cost reduction. Deep analysis of this AI recursive self-optimization event's technical plausibility, industry impact, and key questions.

Jul 29, 2026·3 min

Local LLM Agent Lagging? Lowering num_gpu Is the Key

Local LLM crashing in Agent frameworks? The issue may be num_gpu set too high. Learn what num_gpu really controls (GPU layer offloading, not GPU count) and how to tune it for stable Agent performance.

Jul 29, 2026·3 min

Production-Grade LLM Inference Deployment: A Practical K8s Handbook

Complete guide to deploying production-grade LLM inference on Kubernetes, covering GPU scheduling, vLLM engine selection, autoscaling, observability, and cost optimization.

Jul 29, 2026·3 min

Is LLM Inference Actually Profitable? Breaking Down the Cost Structure and Business Logic of Large Model Inference

A deep dive into LLM inference cost structure and profitability models—from GPU throughput, MoE architecture, and KV Cache to scale effects—revealing the business logic behind API price wars.

Jul 29, 2026·3 min

Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention

Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.

Jul 29, 2026·3 min

Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention

Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.

Jul 29, 2026·2 min

Reverse Engineering Apple's ANE: Open-Source Project Enables On-Device Training on the Neural Engine

Deep dive into the maderix/ANE GitHub project that reverse engineers Apple's private APIs to enable neural network training on the Apple Neural Engine, exploring its technical approach, efficiency gains, compliance risks, and implications for on-device AI.

Jul 29, 2026·2 min

AI Bubble Fears Intensify: The Logic Behind Chip Stocks Selling Off Across U.S. and Asian Markets

Chip stocks fall simultaneously across U.S. and Asian markets as AI bubble fears intensify. Analysis of the drivers, sustainability of AI capex, and the balance between short-term volatility and long-term trends.

Jul 29, 2026·2 min

AI Bubble Fears Intensify: The Logic Behind Chip Stocks Selling Off Across US and Asian Markets

Chip stocks decline simultaneously across US and Asian markets as AI bubble fears intensify. Analysis of the logic behind the selloff, sustainability questions around AI capex, and the relationship between short-term volatility and long-term trends.

Jul 28, 2026·3 min

Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention

A deep dive into Kimi Delta Attention (KDA): tracing the evolution from quadratic Softmax attention through linear attention, Delta rules, and gated decay mechanisms, with insights on associative memory and hardware optimization.

Jul 28, 2026·3 min

Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention

Deep dive into Kimi Delta Attention (KDA): from standard Softmax attention's quadratic bottleneck through linear attention, Delta Rule, and gated decay mechanisms — the complete evolution explained.

Jul 28, 2026·3 min

ncnn Vulkan Cross-Platform Edge Inference in Practice: Solving the GPU Fragmentation Deployment Challenge

Facing GPU fragmentation on edge devices, the PostSlate team used ncnn's Vulkan backend for cross-platform ML inference, achieving 10× speedup on RTX 4070 with half the model size and zero runtime installation.

Jul 28, 2026·3 min

Kimi K3 Launch: A 2.8 Trillion Parameter Flagship Model Makes Its Grand Entrance

Moonshot AI launches Kimi K3 with 2.8 trillion parameters and 1M token context. Google delays Gemini 3.5 Pro, AI coding tools upgrade collectively as competition shifts to coding and Agent capabilities.