#HBM
155 related articles
·3 min
Running Local LLMs on AMD GPUs: Ditching Ollama for LM Studio Delivers 7x Speed Boost
AMD GPU black screens running local LLMs? This post-mortem covers Ollama's 3 fatal flaws and how switching to LM Studio boosted token speed from 5 to 36, with ROCm setup, Speculative Decoding, and GFX version tips.
Read more →
OpenAI and Broadcom Unveil Jalapeño Ch…
·2 min
OpenAI and Broadcom Unveil Jalapeño Chip: Built Exclusively for LLM Inference
OpenAI and Broadcom unveil Jalapeño, a custom ASIC designed for LLM inference. A deep dive into its technical logic, strategic intent, and impact on NVIDIA and the AI compute landscape.
Read more →
·6 min
OpenAI CFO Deep Dive: IPO Stance, Compute Shortage, and Jony Ive's New Hardware Revealed
OpenAI CFO Sarah Fryer discusses the $122B fundraise, IPO timeline, Anthropic rivalry, compute shortage crisis, and the mysterious Jony Ive hardware collaboration on the All-In Podcast.
Read more →
·2 min
Groq Raises $650 Million to Rebuild Team, Challenging NVIDIA with LPU Chips
AI inference chip company Groq confirms $650M funding round, actively rebuilds executive team after NVIDIA's massive talent raid, and doubles down on Neocloud business.
Read more →
·3 min
Making LLMs Faster and Lighter: A Practical Approach to Reshaping Sparsity for GPUs
Deep dive into Sakana AI and NVIDIA's latest research using TwELL sparse packing format and custom CUDA kernels to convert LLM sparsity into real GPU speedups, achieving 20%+ faster inference/training and significantly lower memory usage.
Read more →
·4 min
KV Cache Saves 20x on Costs: The Underlying Principles and Practical Tips for LLM Inference Optimization
Deep dive into how KV Cache reduces LLM API costs by 20x. From Transformer attention matrix multiplication overhead to prompt caching best practices, understand the fundamentals of AI inference cost optimization.
Read more →
·4 min
GLM 5.2 & Zcode Hands-On Review: A Deep Dive into the Free AI Coding Tool with 5 Million Tokens/Day
In-depth review of Zhipu's GLM 5.2 model and Zcode programming tool: interface experience, coding benchmarks, and long-horizon Agent performance compared to GPT and Opus. 5M free tokens/day with MIT license.
Read more →
·2 min
The Compute Crisis: Why Google and Anthropic Are Paying SpaceX a Premium to Rent GPUs
Microsoft, Google, and Anthropic face severe compute shortages. Anthropic pays SpaceX $1B/month for GPUs. From TSMC capacity to HBM, storage, and power, the AI supply chain is in full crisis.
Read more →
·4 min
Local Deployment of Claude Code: Principles and Practical Guide for Three Approaches
A detailed guide to locally deploying Claude Code with three approaches (LM Studio, Ollama, vLLM), covering architecture, protocol translation, hardware selection, and model recommendations.
Read more →
·2 min
A Complete Guide to LLM Infrastructure: Core Challenges from GPU Clusters to Inference Optimization
A deep dive into core challenges and key technologies for LLM infrastructure, covering GPU cluster management, inference optimization, distributed training, cost control, and observability.
Read more →
·4 min
Getting Started with Huawei Ascend C Operator Programming: Kernel Functions and Hands-On Development Guide
A systematic guide to Huawei Ascend C operator programming covering kernel functions, three-stage pipeline paradigm, API categories, and a hands-on AddCustom operator walkthrough.
Read more →
·2 min
Marvell: The Hidden Giant Behind AI Infrastructure — Four Technical Aces Explained
Deep dive into how Marvell leverages UALink switch chips, CXL memory tech, custom ASIC foundry services, and silicon photonics to become an indispensable core supplier in AI infrastructure.
Read more →
·3 min
vLLM Deep Dive: How PagedAttention Enables High-Throughput LLM Inference
Deep dive into vLLM's core technologies for high-throughput LLM inference, including PagedAttention memory management, continuous batching, distributed deployment, and comparisons with TensorRT-LLM.
Read more →
·2 min
AMD Stock Breaks $500: A Deep Dive into the AI Chip Competitive Landscape
AMD stock breaks $500, hitting a new all-time high. Deep analysis of AMD's AI chip strategy, competition with NVIDIA, MI300X advantages, and potential risks.
Read more →
Tutorials·3 min
Agent Tuning: A Complete Guide to Training LLMs with Agent Capabilities
A deep dive into Agent Tuning principles and practices, covering why Agent training is needed, the evolution from Prompt to RAG to Agent, development workflows, and cost assessment for private deployment.
Read more →
Tutorials·3 min
Fully Automated Invoice Reimbursement with Local AI Agent: From Process Breakdown to Deployment
How to build a fully automated invoice reimbursement system with local AI Agents, covering OCR, info extraction, and form generation with MinerU+Qwen3+Qianwen Po.
Read more →
Industry Insights·3 min
China's Internet Giants Collectively Expand AI Capital Expenditure: Six Key Beneficiary Sectors in the Computing Infrastructure Supply Chain
China's internet giants collectively increase AI CapEx as computing infrastructure shifts from expectations to delivery. Analysis of six key beneficiary sectors including AI data centers, chips, and storage.
Read more →
Deep Dives·4 min
Core Principles of the Transformer Architecture: A Deep Dive into Self-Attention Mechanisms and Engineering Optimizations
Deep dive into Transformer architecture covering self-attention QKV mechanics, Encoder-Decoder structure, Flash Attention memory optimization, RoPE positional encoding, and GQA inference acceleration.
Read more →
Tech Frontiers·3 min
OpenAI Codex Deep Dive: AI Programming Evolves from Code Completion to Full-Repository Autonomous Development
Deep analysis of OpenAI Codex agent and its Codex-1 model, comparing Windsurf's SW1E1 and Anthropic's new reasoning model, exploring how AI programming leaps from single-file assistance to full-repository autonomous understanding.
Read more →
Tutorials·2 min
Tutorial: Deploying a PD-Disaggregated SGLang Multi-Node Inference Cluster on AMD GPUs
Learn how to deploy a PD-disaggregated SGLang inference cluster on AMD GPUs using a single config file, boosting LLM throughput and latency performance.
Read more →