#GPU memory bottleneck

390 related articles

Jul 10, 2026·5 min

TensorRT Multi-Device Inference: Engineering Practices for Breaking Through Single-GPU Memory Bottlenecks

NVIDIA TensorRT now supports multi-device inference via pipeline and tensor parallelism, distributing large models across multiple GPUs to break through single-card memory limits.

NVIDIA GQE Deep Dive: How GPU Query En…

Jul 7, 2026·2 min

NVIDIA GQE Deep Dive: How GPU Query Engines Break Through Memory Bandwidth Bottlenecks

A deep dive into NVIDIA GQE's architecture: how HBM, NVLink, and memory-hierarchy-aware execution models help GPU query engines overcome I/O and bandwidth bottlenecks.

Jul 29, 2026·2 min

Godot Engine VR Development Log: Lessons and Pitfalls from Porting to PSVR2

An in-depth analysis of an indie developer's experience using Godot to develop VR games and port to PSVR2, covering OpenXR integration, performance optimization, and console certification challenges.

Jul 29, 2026·3 min

Production-Grade LLM Inference Deployment: A Practical K8s Handbook

Complete guide to deploying production-grade LLM inference on Kubernetes, covering GPU scheduling, vLLM engine selection, autoscaling, observability, and cost optimization.

Jul 29, 2026·3 min

Is LLM Inference Actually Profitable? Breaking Down the Cost Structure and Business Logic of Large Model Inference

A deep dive into LLM inference cost structure and profitability models—from GPU throughput, MoE architecture, and KV Cache to scale effects—revealing the business logic behind API price wars.

Jul 29, 2026·3 min

Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention

Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.

Jul 29, 2026·3 min

Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention

Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.

Jul 29, 2026·3 min

GitHub Daily · July 29: Voice AI and On-Device Training Take Flight as Open-Source Alternatives Rise

GitHub Trending July 29: Microsoft's VibeVoice leads voice AI open-source wave, MoonshotAI's FlashKDA CUDA kernel surges 25%, and open-source alternatives rise.

Jul 29, 2026·4 min

Project Rai-chan Technical Breakdown: A Complete Guide to Building a Fully Local AI Companion

Deep dive into Project Rai-chan's tech stack: Ollama+Gemma local LLM, Unity rendering, VOICEVOX speech synthesis, and more — exploring the technical path for local AI companions.

Jul 28, 2026·2 min

Training AI to Play Devil May Cry 3 with Reinforcement Learning: Challenges and Practices in Action Game AI

An indie developer trains AI to autonomously play Devil May Cry 3 using reinforcement learning. Explore the core challenges of action game AI including sparse rewards, high-dimensional action spaces, and real-time decision-making.

Jul 28, 2026·2 min

Training AI to Play Devil May Cry 3 with Reinforcement Learning: Challenges and Practices for Action Game AI

An indie developer trains AI to autonomously play Devil May Cry 3 using reinforcement learning. This article analyzes the core challenges including sparse rewards, high-dimensional action spaces, and real-time decision-making.

Jul 28, 2026·3 min

Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention

A deep dive into Kimi Delta Attention (KDA): tracing the evolution from quadratic Softmax attention through linear attention, Delta rules, and gated decay mechanisms, with insights on associative memory and hardware optimization.

Jul 28, 2026·3 min

Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention

Deep dive into Kimi Delta Attention (KDA): from standard Softmax attention's quadratic bottleneck through linear attention, Delta Rule, and gated decay mechanisms — the complete evolution explained.

Jul 28, 2026·3 min

Kimi K3 Launch: A 2.8 Trillion Parameter Flagship Model Makes Its Grand Entrance

Moonshot AI launches Kimi K3 with 2.8 trillion parameters and 1M token context. Google delays Gemini 3.5 Pro, AI coding tools upgrade collectively as competition shifts to coding and Agent capabilities.

Jul 28, 2026·3 min

Kimi K3 Open-Source Release: 2.8 Trillion Parameters Tops Global Programming Leaderboard

Moonshot AI's Kimi K3 launches with 2.8 trillion parameters, tops LMArena frontend coding leaderboard as world #1, completing tasks at one-third competitors' cost. Fully open-source for commercial use.

Jul 28, 2026·5 min

July 24 AI Daily Brief: Flux 3 Multimodal Release, Kimi K3 Lags in Testing, Qwen Tops TTS Rankings

July 24 AI news: Black Forest Labs launches Flux 3 multimodal model, Kimi K3 lags in US-UK gov tests, Alibaba Qwen tops TTS rankings, Etched raises $300M, AMD unveils MI430X.

Jul 28, 2026·4 min

Kimi K3 Released: 2.8 Trillion Parameter Open-Source Model Tops Global Coding Leaderboard

Moonshot AI releases Kimi K3, a 2.8 trillion parameter open-source model using MoE architecture that tops the global frontend coding arena at under $1 per task, beating GPT and Claude.

Jul 28, 2026·3 min

Backend Developer Transition to AI/LLM: 4 Core Skills and a Practical Learning Roadmap

Complete guide for backend developers transitioning to AI/LLM engineering. Covers the 4 core skills—Python, RAG, Fine-tuning, and Agents—with a phased learning roadmap and practical project advice.

Jul 28, 2026·4 min

Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop

Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.

Jul 28, 2026·4 min

Laguna S 2.1 Deep Dive: Hands-On Local Deployment of a 118B Parameter Coding Model

In-depth review of Poolside's Laguna S 2.1 open-source coding model: MoE architecture, RL training, DGX Spark local deployment, and real-world agentic coding tests with 8B active parameters.