#GPU throughput

431 related articles

Jul 27, 2026·3 min

Opus 5 Hands-On: Powerful but Slow — Inference Speed Becomes the New AI Battleground

An AI practitioner tests Opus 5: powerful capabilities but painfully slow inference. Deep analysis of how LLM competition is shifting from intelligence to inference speed.

Jul 27, 2026·3 min

HuggingHack Update: S3 Storage, Dual-Engine Scheduling, and GGUF Inspection Get Major Upgrades

HuggingHack releases major updates with S3/MinIO storage, Ollama + vLLM dual-engine scheduling, GGUF inspection, and local accounts for enterprise-grade local LLM management.

Jul 27, 2026·4 min

FPGA Softmax Approximation in Practice: A Detailed Guide to Taylor Expansion and Padé Approximation Methods

A detailed guide on efficiently implementing Softmax on FPGAs, comparing Taylor series and Padé approximation methods for accuracy and resource trade-offs, with Python simulation and range reduction techniques.

Jul 27, 2026·7 min

DIY Companion Robot: Building Your AI Desktop Buddy with Jetson Orin

A maker builds a DIY companion robot with NVIDIA Jetson Orin and 4S LiPo battery. Explore the full development journey from first power-up to AI interaction, including edge computing, power design, and companion robot trends.

Jul 26, 2026·3 min

Rust and Mojo Are Quietly Replacing Python's Low-Level Engine

Python tops the language rankings again, but AI teams are quietly swapping its internals for Rust and Mojo. A look at Python's speed and GIL pains, the two-language problem, and the rise of Rust tooling and Mojo on GPUs.

Jul 26, 2026·3 min

DeepSeek-OCR Deployment and Fine-Tuning: A Complete Hands-On Guide

Master the full DeepSeek-OCR deployment and fine-tuning workflow: vLLM inference deployment, efficient Unsloth fine-tuning, dataset preprocessing, LoRA training, validation, and RAG vector database integration.

Jul 25, 2026·2 min

DSpark: DeepSeek Partners with Peking University to Open-Source Inference Acceleration, Boosting Speed by Up to 85%

DeepSeek partners with Peking University to open-source DSpark, an inference acceleration tech boosting single-user speed by 57%-85% under high concurrency. Learn its three core designs and the DSpec framework.

Jul 25, 2026·2 min

DSpark: DeepSeek and Peking University Open-Source Inference Acceleration Tech with Up to 85% Speed Boost

DeepSeek and Peking University open-source DSpark, an inference acceleration tech boosting single-user generation speed by 57%-85% under high concurrency. Learn its 3 core designs and the DSpec framework.

Jul 25, 2026·2 min

D-Flash: How Diffusion Drafting Delivers 3.5x Faster LLM Inference

D-Flash solves the autoregressive drafter latency bottleneck in speculative decoding via fast diffusion parallel drafting and target feature KV injection. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, beating EAGLE3 and MTP.

Jul 25, 2026·3 min

D-Flash: The Secret Behind 3.5x LLM Inference Speedup with Diffusion Drafting

D-Flash uses fast diffusion parallel drafting and target feature KV injection to solve the autoregressive Drafter latency bottleneck in speculative decoding. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, surpassing EAGLE3 and MTP.

Jul 25, 2026·3 min

Laguna S 2.1: The Open-Source Coding Model That's 14x Smaller Yet Stronger

Poolside releases Laguna S 2.1, an open-source agentic coding model: 118B total params with only 8B active, scoring 40.4% on DeepSWE — ~4.5x DeepSeek V4 Pro Max. Supports 1M-token context, deployable on a single workstation.

Jul 25, 2026·3 min

DeepSeek Open-Sources DeepSpec: A Full-Stack Speculative Decoding Library with 60–85% Inference Speedup

DeepSeek open-sources DeepSpec, a full-stack speculative decoding library delivering 60–85% inference speedup and up to 661% throughput gains in production. MIT license, 9 checkpoints, 9 benchmarks.

Jul 25, 2026·2 min

The New Bar for AI/LLM Roles: A Practical Guide to Agent Observability and Evaluation

Enterprise AI/LLM roles now demand engineering skills: streaming recovery, high concurrency, multi-tenancy, LLM gateways, Langfuse observability, and evaluation platforms. Master these 8 core competencies.

Jul 24, 2026·3 min

OpenAI Releases GPT-5.6 and ChatGPT Work: The Era of Autonomous AI Has Arrived

OpenAI launches GPT-5.6 (Sol, Terra, Luna), ChatGPT Work, a new desktop app, and Sites hosting. AI evolves from a Q&A tool into an autonomous work partner for finance, file management, and more.

Jul 24, 2026·6 min

AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment

A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.

Jul 24, 2026·6 min

AI Application Engineer Interview Guide: Core Topics on Model Quantization and Inference Deployment

A systematic guide to must-know AI application engineer interview topics: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.

Jul 24, 2026·6 min

The AI Industrial Age Arrives: Compute Wars, Chip Reconstruction, and the GPT-5.6 Cheating Scandal Explained

Five key AI industry trends: Doubao surpasses 180 trillion daily calls, OpenAI's in-house AI chip, NVIDIA's $3-4 trillion compute forecast, China catching up, and the GPT-5.6 cheating scandal.

Jul 24, 2026·6 min

Hands-On with Panel AI On-Premises Deployment: One-Click Enterprise Adoption and Unified Compute Cluster Scheduling

In-depth review of Panel AI v1.1.1: second-level installation, no-public-IP networking, batch compute cluster management. Learn how enterprise AI on-premises deployment barriers are dramatically lowered.

Jul 24, 2026·5 min

China Weighs Restricting Top AI Model Exports as DeepSeek Develops Its Own Inference Chip

China's Commerce Ministry weighs restricting overseas access to top open-weight AI models from Alibaba, ByteDance, and DeepSeek, while DeepSeek develops its own inference chip amid a US-China AI export standoff.

Jul 24, 2026·5 min

China Weighs Restricting Top AI Model Exports, While DeepSeek Quietly Builds Its Own Inference Chip

China's Ministry of Commerce is weighing restrictions on overseas access to top open-weight AI models from Alibaba, ByteDance, and DeepSeek. Meanwhile, DeepSeek is quietly building its own inference chip and raising ~$7B in first external funding.