#LLM inference acceleration

198 related articles

Jul 17, 2026·4 min

DSpark Open Source: DeepSeek's Inference Acceleration Tech That Doubles LLM Speed

DeepSeek and Peking University open-source DSpark, an inference acceleration technology using semi-autoregressive architecture and dynamic scheduling to boost LLM speed by 50%+ and double GPU concurrency without quality loss.

Jul 28, 2026·3 min

Spring AI Explained: Unified API, RAG, and Enterprise AI Development Guide

Deep dive into Spring AI framework's core features including provider-agnostic unified API abstraction, RAG retrieval-augmented generation, and structured output to help Java developers build enterprise AI apps.

Jul 28, 2026·3 min

Spring AI Explained: Unified API, RAG, and Enterprise AI Development Practical Guide

Jul 28, 2026·4 min

Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop

Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.

Jul 28, 2026·3 min

A Beginner's Guide to AI Agents: Two Learning Paths for Developers and Practitioners

How to learn AI Agents from scratch? This guide covers two clear paths: developers go from Python to LLMs to open-source framework source code; practitioners use Claude Code or similar tools to get results fast.

Jul 28, 2026·3 min

Google Launches Gemini 4 Pre-training: Its Most Ambitious AI Model Training Yet

Google DeepMind announces Gemini 4 pre-training has begun, calling it their most ambitious training yet. A deep dive into its technical direction, compute scale, multimodal breakthroughs, and competitive impact.

Jul 27, 2026·2 min

RX 9060 XT vs RTX 5060 Ti: Which GPU to Pick for Local AI?

RX 9060 XT vs RTX 5060 Ti — both 16GB VRAM, but which is better for local AI? We compare CUDA ecosystem, ROCm compatibility, LLM inference, and real-world usability.

Jul 27, 2026·2 min

RX 9060 XT vs RTX 5060 Ti: How to Choose a GPU for Local AI

RX 9060 XT vs RTX 5060 Ti both offer 16GB VRAM — which is better for local AI inference? A full comparison of CUDA ecosystem, ROCm compatibility, LLM performance, and real-world usability.

Jul 27, 2026·2 min

Kimi-K3 Lands on HuggingFace: A Full Breakdown of Moonshot AI's Open-Source Strategy

Moonshot AI's Kimi-K3 model is set to launch on HuggingFace. A deep dive into its architecture, licensing, community reactions, and strategic implications for the global open-source AI landscape.

Jul 27, 2026·3 min

Opus 5 Hands-On: Powerful but Slow — Inference Speed Becomes the New AI Battleground

An AI practitioner tests Opus 5: powerful capabilities but painfully slow inference. Deep analysis of how LLM competition is shifting from intelligence to inference speed.

Jul 27, 2026·4 min

Gemini 3.5 Checkpoint Leaked: Intermediate Version Reportedly Outperforms Opus 5 in Testing

Reddit leaks suggest a Google Gemini 3.5 intermediate checkpoint outperformed Claude Opus 5 max thinking in testing. We analyze what checkpoints mean, benchmark credibility, and the LLM competition landscape.

Jul 27, 2026·7 min

DIY Companion Robot: Building Your AI Desktop Buddy with Jetson Orin

A maker builds a DIY companion robot with NVIDIA Jetson Orin and 4S LiPo battery. Explore the full development journey from first power-up to AI interaction, including edge computing, power design, and companion robot trends.

Jul 26, 2026·5 min

From Chat to Agent: Automating Your Entire Business Workflow with AI Agents

Veteran AI practitioner Remy breaks down the leap from chat models to AI agents: how agents work, the three pillars of context, tools, and skills, MCP connections, and hands-on architecture to make you a 100x employee.

Jul 26, 2026·3 min

DeepSeek-OCR Deployment and Fine-Tuning: A Complete Hands-On Guide

Master the full DeepSeek-OCR deployment and fine-tuning workflow: vLLM inference deployment, efficient Unsloth fine-tuning, dataset preprocessing, LoRA training, validation, and RAG vector database integration.

Jul 25, 2026·2 min

DSpark: DeepSeek and Peking University Open-Source Inference Acceleration Tech with Up to 85% Speed Boost

DeepSeek and Peking University open-source DSpark, an inference acceleration tech boosting single-user generation speed by 57%-85% under high concurrency. Learn its 3 core designs and the DSpec framework.

Jul 25, 2026·2 min

D-Flash: How Diffusion Drafting Delivers 3.5x Faster LLM Inference

D-Flash solves the autoregressive drafter latency bottleneck in speculative decoding via fast diffusion parallel drafting and target feature KV injection. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, beating EAGLE3 and MTP.

Jul 25, 2026·3 min

D-Flash: The Secret Behind 3.5x LLM Inference Speedup with Diffusion Drafting

D-Flash uses fast diffusion parallel drafting and target feature KV injection to solve the autoregressive Drafter latency bottleneck in speculative decoding. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, surpassing EAGLE3 and MTP.

GitHub Daily · July 25: Where Decentra…

Jul 25, 2026·2 min

GitHub Daily · July 25: Where Decentralized Communication Meets AI-Native Tools

GitHub Trending July 25: bitchat's Bluetooth Mesh chat gains 1,700 stars in a day, palmier-pro reimagines AI-native video editing, and aisuite unifies LLM access.

Jul 25, 2026·3 min

Laguna S 2.1: The Open-Source Coding Model That's 14x Smaller Yet Stronger

Poolside releases Laguna S 2.1, an open-source agentic coding model: 118B total params with only 8B active, scoring 40.4% on DeepSWE — ~4.5x DeepSeek V4 Pro Max. Supports 1M-token context, deployable on a single workstation.

Jul 25, 2026·3 min

DeepSeek Open-Sources DeepSpec: A Full-Stack Speculative Decoding Library with 60–85% Inference Speedup

DeepSeek open-sources DeepSpec, a full-stack speculative decoding library delivering 60–85% inference speedup and up to 661% throughput gains in production. MIT license, 9 checkpoints, 9 benchmarks.