#GPU

1540 related articles

Jul 30, 2026·3 min

Is LLM Inference Really That Expensive? Breaking Down the Cost Reduction Logic of GPU Compute and Inference Frameworks

Analyzing real LLM inference costs: from B200 GPU compute gains, vLLM framework optimization to MTP multi-token prediction, explaining why serving costs are widely overestimated.

Jul 29, 2026·3 min

Local LLM Agent Lagging? Lowering num_gpu Is the Key

Local LLM crashing in Agent frameworks? The issue may be num_gpu set too high. Learn what num_gpu really controls (GPU layer offloading, not GPU count) and how to tune it for stable Agent performance.

Jul 29, 2026·4 min

9 Hours, 4 Million Tokens: AI Creates a Stunning WebGPU Snow Rendering Masterpiece with Claude Code

Reddit user Noniv used Claude Code with the Opus model to build SNOWFLOW, a stunning WebGPU real-time snow renderer, in just 9 hours using 4 million tokens. A deep dive into prompt strategy, snow shaders, and performance optimization.

Jul 29, 2026·2 min

Running Manim in the Browser: Rebuilding 3Blue1Brown's Animation Engine with WebGPU

Explore how WebGPU brings 3Blue1Brown's Manim math animation engine to the browser, enabling zero-install, real-time mathematical visualization with interactive education potential.

Jul 28, 2026·3 min

ncnn Vulkan Cross-Platform Edge Inference in Practice: Solving the GPU Fragmentation Deployment Challenge

Facing GPU fragmentation on edge devices, the PostSlate team used ncnn's Vulkan backend for cross-platform ML inference, achieving 10× speedup on RTX 4070 with half the model size and zero runtime installation.

Jul 27, 2026·2 min

RX 9060 XT vs RTX 5060 Ti: How to Choose a GPU for Local AI

RX 9060 XT vs RTX 5060 Ti both offer 16GB VRAM — which is better for local AI inference? A full comparison of CUDA ecosystem, ROCm compatibility, LLM performance, and real-world usability.

Jul 27, 2026·2 min

RX 9060 XT vs RTX 5060 Ti: Which GPU to Pick for Local AI?

RX 9060 XT vs RTX 5060 Ti — both 16GB VRAM, but which is better for local AI? We compare CUDA ecosystem, ROCm compatibility, LLM inference, and real-world usability.

Jul 27, 2026·2 min

Behind NVIDIA's Investment in SSI: The GPU Demand Self-Reinforcing Loop and the Superintelligence Gamble

NVIDIA's investment in Ilya Sutskever's SSI reveals a GPU demand self-reinforcing loop, a secretive superintelligence strategy, and the deep entanglement of AI capital and technology.

Jul 25, 2026·5 min

DeepSeek's Five-Step Ladder to AGI: A Restrained Roadmap Built on 20,000 GPUs

DeepSeek founder Liang Wenfeng reveals a five-step AGI roadmap—from chain-of-thought to embodied intelligence—under a 20,000-GPU constraint, using the TileLang compiler to break domestic substitution challenges while API cash flow backs AGI exploration.

Jul 25, 2026·5 min

DeepSeek's Five-Step Ladder to AGI: A Restrained Roadmap Built on Twenty Thousand GPUs

DeepSeek founder Liang Wenfeng reveals a five-step AGI roadmap from chain-of-thought to embodied intelligence. How does TileLang crack domestic GPU substitution under a 20,000-card constraint?

Jul 22, 2026·3 min

V100 GPU Showdown: 4×16G PCIe vs. 2×32G SXM — Which Gives You More Bang for Your Buck?

Benchmarking 4×V100 16G PCIe vs. 2×V100 32G SXM adapter for local LLM inference. Prefill speed, decode speed, power limits, and bandwidth bottlenecks analyzed.

Jul 20, 2026·2 min

On-Device LLMs, Custom Silicon, and 100K-GPU Clusters: AI Industry Enters Full-Spectrum Competition

Apple's 1-bit compression fits 27B models on iPhone, Meta builds custom chip Iris, China's 100K-GPU cluster goes live, Samsung enters AI PC — a deep dive into AI's new full-spectrum competition.

AirLLM: How a 4GB GPU Can Run a 70B Mo…

Jul 19, 2026·2 min

AirLLM: How a 4GB GPU Can Run a 70B Model — Core Principles and Use Cases

AirLLM is an open-source project that uses layer-by-layer inference to run 70B LLMs on a single 4GB GPU. Learn how it works, its tradeoffs, and ideal use cases.

Block Low-Rank Compression: A Guide to…

Jul 19, 2026·2 min

Block Low-Rank Compression: A Guide to Accelerating Large Model Inference on Memory-Constrained GPUs

Learn how Block Low-Rank (BLR) decomposition compresses large model memory usage and accelerates GPU inference, including CUDA kernel optimization and combination with quantization and pruning.

CUDA 13.3 Adds Carryless Multiplicatio…

Jul 17, 2026·2 min

CUDA 13.3 Adds Carryless Multiplication: A Key Step Forward for GPU Cryptography Acceleration

CUDA 13.3 introduces native carryless multiplication support, closing a 15-year GPU gap in AES-GCM, CRC, and cryptographic acceleration. Here's what it means.

Jul 16, 2026·3 min

GPUStack in Action: Complete Guide to Day 0 Deployment of DeepSeek-V4-Flash

Deploy DeepSeek-V4-Flash DSpark on 8× H20-141G using GPUStack's SGLang backend on Day 0. Full walkthrough of Web UI config, parameter tuning, and 200 tokens/sec benchmark results.

Jul 16, 2026·2 min

GPU Sharing in Kubernetes Production: A Deep Dive into MIG, MPS, and Time-Slicing

A deep dive into MIG, MPS, and Time-Slicing GPU sharing solutions for Kubernetes production. Compare isolation, performance, and use cases to make the right choice.

Jul 16, 2026·3 min

GPU Rental vs. Buying: How Low-Usage Teams Should Actually Do the Math

Should low-usage teams rent or buy GPUs? Based on real billing data, we break down hardware costs, electricity, and hidden time losses to help small teams decide.

Jul 15, 2026·2 min

Womprat: Open-Source Small Aerial Target Tracking Without a GPU

Womprat is an open-source lightweight aerial target tracking tool by Punk Labs. Runs fully on CPU — no GPU or trained models required. Uses classical CV algorithms.

Jul 15, 2026·2 min

Mesh LLM: Pool Multiple Old Computers Into a Virtual Super GPU for Running Large Models

Mesh LLM is an open-source distributed inference framework that splits model layers across multiple devices, creating a virtual super GPU to run 100GB+ LLMs on consumer hardware.