#speculative decoding

175 related articles

Jul 21, 2026·2 min

GPT-Live Full-Duplex Voice Model: Listen While Speaking + Reasoning Delegation Architecture Explained

OpenAI's GPT-Live full-duplex voice model enables natural simultaneous conversation with a reasoning delegation architecture pairing real-time dialogue with GPT-5.5 deep reasoning. Now live for 150M users.

Jul 21, 2026·3 min

What Is a Large Language Model, Really? The Core Principles Explained Through the Lens of Functions

Demystify large language models using middle-school math: LLMs are complex functions, training solves for parameters, and inference predicts next-token probabilities.

Jul 20, 2026·2 min

GPT-Live Real-Time Voice Interaction: How "Listen and Speak Simultaneously" Enables Natural Conversation

GPT-Live breaks the "turn-based" limit with millisecond real-time voice interaction. Explore the tech behind simultaneous listen-and-speak, multilingual interpretation demos, and AI voice's shift from tool to conversational partner.

Jul 20, 2026·3 min

Running a 27B Model on iPhone: How 1-bit Quantization Achieves Extreme Compression

PrismML's Bonsai compresses a 27B model from 54GB to 3.9GB, running at ~11 tokens/sec on iPhone. A deep dive into QAT, knowledge distillation, and speculative decoding.

Jul 17, 2026·4 min

DSpark Open Source: DeepSeek's Inference Acceleration Tech That Doubles LLM Speed

DeepSeek and Peking University open-source DSpark, an inference acceleration technology using semi-autoregressive architecture and dynamic scheduling to boost LLM speed by 50%+ and double GPU concurrency without quality loss.

Jul 16, 2026·2 min

OpenAI Reportedly Planning Major Price Cuts: A Direct Strike at Anthropic's API Market

OpenAI is reportedly in internal discussions about major API token price cuts targeting Anthropic's user base. We analyze the market logic, developer impact, and where AI infrastructure competition is headed.

Jul 16, 2026·3 min

LongCat 2.0: Meituan Trains a 1.6 Trillion Parameter LLM on Domestic Chips

Meituan open-sources LongCat 2.0, a 1.6T-parameter MoE model trained on 50,000+ custom chips without NVIDIA GPUs or Google TPUs, rivaling OpenAI and Google.

Jul 16, 2026·3 min

DeepSeek Open-Sources DSpark: 50%–400% LLM Inference Speedup Without Retraining

DeepSeek open-sources DSpark, delivering 50%–400% LLM inference speedup with no retraining or quantization, via semi-autoregressive drafting and confidence-scheduled verification.

Jul 15, 2026·3 min

Perplexity AI Deep Dive: The Perfect Fusion of Search Engine and AI Q&A

Deep dive into Perplexity AI: real-time web search + multi-model AI, transparent citations, Focus Modes, PDF chat, and collaborative Collections. Can it replace Google and ChatGPT?

Jul 15, 2026·4 min

Running Qwen3.6 27B Locally on a Single RTX 3090: A Complete Guide to Cloud-Rivaling Performance

One used RTX 3090, one 16.8GB GGUF file, and Qwen3.6 27B runs locally offline. SWE-bench score of 77 rivals Claude Sonnet. MTP boosts speed to 59 tok/s. Full local AI coding assistant deployment guide.

Jul 15, 2026·3 min

Semantic Caching: The Core Technology and Practical Guide to Cutting AI Costs by 50%

Semantic caching uses vector embeddings to match similar queries and avoid redundant LLM calls, cutting AI app costs by up to 50%. Learn how it works, how to set thresholds, and when to use it.

Jul 15, 2026·4 min

Two Paths to Better AI Reasoning: Test-Time Scaling and Formal Verification

This article synthesizes two MSR India Summit talks, exploring two key paths to better AI reasoning: test-time scaling with variable granularity search, and a formal verification framework for trustworthy agent execution.

Jul 13, 2026·3 min

Distributed AI Systems in Practice: A Complete Guide to Training, Inference, and Production Deployment

A deep dive into Distributed AI Systems: a new book distilling 10 years of AI engineering experience covering distributed training, inference optimization, and production model serving.

Jul 13, 2026·2 min

Grok 4.5 Hands-On Review: Speed, Accuracy, and Value in a Three-Way Evaluation

A power-user developer tests Grok 4.5: blazing-fast responses, high accuracy, and great value. A deep dive into Grok 4.5's real-world AI coding performance in Cursor.

Jul 13, 2026·4 min

AI Weekly Recap: Four Major Vendors Cut Prices Together, Copilot's Paid Conversion Rate Just 4.5%

In one week, OpenAI, xAI, Google, and Microsoft all cut AI prices, driving near-frontier inference costs sharply lower. Meanwhile, Microsoft Copilot's paid conversion across 450M seats is under 4.5%, exposing the monetization challenge of general AI assistants.

Jul 12, 2026·4 min

GPT-5.6 Released: Coding Benchmarks Surpass Rivals, Inference Speed 10x Faster

Just 12 days after a rival launch, OpenAI released GPT-5.6, scoring 91.9% on Terminal-Bench 2.1 to surpass competitors. Ultra mode supports multi-agent collaboration, inference hits 750 tokens/sec.

Jul 12, 2026·4 min

Unsloth Releases Qwen3.6 NVFP4 Quantization: 2.5x Faster with Zero Accuracy Loss

Unsloth releases NVFP4 quantization for Qwen3.6 using W4A4 true 4-bit Tensor Core computation, delivering up to 2.5x inference speedup over NVIDIA's official implementation with accuracy matching or exceeding BF16 on benchmarks like MMLU-Pro.

Jul 12, 2026·4 min

OpenAI GPT-Live Voice Model: How Full-Duplex Interaction Reshapes Human-AI Conversation

OpenAI unveils the GPT-Live voice model family, with full-duplex interaction enabling AI to listen and speak simultaneously and delegate complex reasoning to GPT-5.5. GPQA benchmark jumps from 45% to 80%.

Jul 12, 2026·5 min

GPT-5.6 Released: Hands-On Analysis of Programmatic Tool Calling and Autonomous Agents

GPT-5.6 is officially released with core upgrades including programmatic tool calling, autonomous subagent delegation, and higher token information density. A hands-on card game build reveals its Agentic power.

Local Coding Agents in Practice: A Com…

Jul 12, 2026·6 min

Local Coding Agents in Practice: A Complete Guide to Replacing Your Claude Subscription with Open-Source Models

An in-depth look at local coding agents—core concepts, advantages, and real challenges. Compare against Claude Code and learn to build a zero-subscription, private AI coding workflow with open-weight models.