#autoregressive decoding
178 related articles
·2 min
omlx: High-Performance LLM Inference Server Built for Apple Silicon
omlx is an open-source LLM inference server optimized for Apple Silicon, featuring continuous batching, SSD caching, and macOS menu bar management.
Read more →
·3 min
Netflix GenRec: How an LLM-Native Recommendation System Is Reshaping Personalized Recommendations
Deep analysis of Netflix GenRec's generative recommendation system, covering Semantic IDs, LLM-native architecture, and the paradigm shift from discriminative to generative recommendation.
Read more →
·2 min
OpenAI Employee Reveals ChatGPT Speed Optimization Roadmap
OpenAI employee shares ChatGPT speed improvement roadmap on Reddit, covering inference optimization, model distillation, and infrastructure scaling to reduce response latency.
Read more →
·3 min
Building an LLM Inference Engine in Rust That Rivals Llama.cpp
Exploring the technical path to building an LLM inference engine in pure Rust that rivals Llama.cpp, analyzing Rust's advantages and challenges in memory safety, SIMD optimization, and GPU backends.
Read more →
·3 min
Muse-Glimmer-30B Real-World Test: The Best Bang-for-Buck Model Scoring 76 Points Across 9 Dimensions on Just 24GB VRAM
In-depth test of Meta's Muse-Glimmer-30B: 76.04 avg across 9 dimensions, 90+ tool calling scores, near-lossless 4-bit quantization on 24GB VRAM, and 3.1x D-Flash speedup reaching 233 tokens/sec.
Read more →
·4 min
Hands-On Testing Meta's Open-Source Muse Glimmer: A Lopsided Agent-Specific Model
Hands-on testing of Meta's open-source 30B Muse Glimmer model across vision, reasoning, and full-stack tasks. Excellent vision but weak logic, D-Spark gives 3x speed at quality cost, 128K context is the biggest limitation.
Read more →
·3 min
Open-Source Model Showdown: NVIDIA, Meta, and Alibaba Release in the Same Week — Who Wins?
NVIDIA Nemotron 3.5 Lightning, Meta Muse Glimmer, and Alibaba Qwen 3.8 all launched in the same week. We compare speed, intelligence scores, and local deployment to find the best model for local Agents.
Read more →
·3 min
Meta Open-Sources Muse Glimmer: A 30-Billion Parameter Agent Model That Runs Locally on a Single GPU
Meta open-sources Muse Glimmer, a 30B parameter agent model compressed to under 20GB via 4-bit quantization. Runs on a single RTX 4090 with 128K context, 3x speedup via D-Flash speculative decoding, and MCP tool-calling score of 75.5.
Read more →
·4 min
Understanding RAG from Scratch: Give a Large Model Context and It Can Answer Anything
What is RAG (Retrieval-Augmented Generation)? This article explains RAG core concepts with simple analogies, analyzes three LLM pain points, and details RAG's working mechanism and future trends.
Read more →
·2 min
NVIDIA Open-Sources Switchyard: An AI Task Scheduling Engine Built with Rust
NVIDIA-NeMo team open-sources Switchyard, a high-performance AI task scheduling engine built in Rust. Explore its technical positioning, why Rust was chosen, and its strategic role in the NeMo ecosystem.
Read more →
·3 min
Claude's Invisible Watermark Technology Explained: How AI Text Tracing Works
Anthropic embeds invisible text watermarks in all Claude outputs for machine-readable tracing. Deep dive into the watermark technology, C2PA file provenance standard, and industry impact.
Read more →
·3 min
DeepSeek Plans Significant Price Hikes: Is the Era of Cheap AI Coming to an End?
DeepSeek plans significant API price hikes, signaling the end of ultra-cheap AI. We analyze the drivers, developer impact, and industry shift from price wars to rational pricing.
Read more →
·3 min
Prompt Caching Explained: The Key Technology for Saving Money with AI Agents
Deep dive into Prompt Caching: how it works, why AI Agents repeatedly send tokens causing costs to skyrocket, and best practices to slash LLM costs by up to 90%.
Read more →
·3 min
Latent Space Reasoning: How DeepSeek-V4 Lets AI Think in Hidden Layers
Deep dive into DeepSeek-V4's latent space reasoning technology — how AI shifts from explicit chain-of-thought to implicit vector space reasoning, its efficiency gains, and challenges in interpretability.
Read more →
·2 min
Airy Voice Content Creation Tool: A Free, Fast, and Simple AI Voiceover Solution
Airy is a free, fast, and simple AI voice content creation tool. This article analyzes Airy's positioning, technology trends, market opportunities, and challenges in the lightweight voice creation space.
Read more →
·3 min
Ollama Cloud Unsubscription Story: A Real-World Comparison of GLM vs DeepSeek
A Django developer shares their Ollama Cloud subscription experience, comparing GLM 5.2 and DeepSeek V4 Pro for PHP programming, analyzing cloud AI coding service value for indie developers.
Read more →
·2 min
DiacTag: The Engineering Wisdom of Transforming Diacritic Restoration into a Constrained Classification Task
DiacTag redefines diacritic restoration as constrained classification rather than generation, providing structural guarantees that output never deviates from input through architectural design.
Read more →
·3 min
Token Economics: The New Unit of Value Measurement in the AI Era
Deep dive into how Tokens evolved from a technical concept in LLMs to the core unit of measurement in the AI economy. Exploring Token consumption explosion, cost optimization, and Token economics.
Read more →
·2 min
GPT-5.6 Luna Price Cut by 80%: A Full Breakdown of OpenAI's Latest Pricing Strategy
OpenAI announces major GPT-5.6 price cuts: Luna down 80%, Terra down 20%, Sol gets faster API options. Full analysis of strategy and developer impact.
Read more →
·2 min
CostPerPrompt: A Deep Dive into Real-Time AI API Cost Estimation and Pricing Comparison
CostPerPrompt is a real-time AI API pricing comparison and cost estimation tool supporting OpenAI, Anthropic, Google and more, helping developers estimate monthly token costs based on real workloads.
Read more →