#parallel decoding
167 related articles
Tech Frontiers·1 min
Qwen3.6 MTP Acceleration Benchmarked: Single GPU Inference Hits 220 token/s
Qwen3.6 experimental MTP-GGUF benchmarked: single GPU pushes 35B-A3B model to 220 token/s, 1.4x faster with zero accuracy loss. Covers MTP principles, optimal Draft Tokens strategy, and RTX 5090 results.
Read more →
Product Reviews·1 min
Qwen 3.6 MTP Real-World Test: The Secret to a 20% Speed Boost with Just Three Parameters
Real-world test of Qwen 3.6 Multi-Token Prediction (MTP): boost inference speed from 34.2 to 41 tokens/s with just three parameters in ik_llama.cpp — zero quality loss, zero extra models.
Read more →
Tech Frontiers·4 min
Step 3.5 Flash Deep Dive: How a 196B Parameter Open-Source Model Beat Gemini
Deep dive into StepFun's Step 3.5 Flash: 196B parameter MoE model activating only 11B, 350 tokens/sec coding speed, 256K context window, local deployment ready. How it beats Gemini 3 Flash.
Read more →
Product Reviews·3 min
Manus vs Tiangong vs Liaobots: AI Subtitle Translation Real-World Scoring Comparison
Real-world comparison of Manus, Tiangong, and Liaobots translating English tech presentation subtitles, scored across colloquial handling, terminology accuracy, and ASR error correction.
Read more →
Tutorials·3 min
Grammar-Constrained Decoding: A Practical Guide to Generating Reliable Bash Code with Small Models
Deep dive into Grammar-Constrained Decoding (GCD) technology: applying Bash syntax constraints during inference to dramatically improve small language models' code generation correctness and executability for AI Agent edge deployment.
Read more →
Deep Dives·3 min
NVIDIA Dynamo Multi-Turn Agentic Interaction: Deep Integration of Streaming Tokens and Tool Calling
Deep dive into NVIDIA Dynamo's multi-turn agentic interaction support, covering streaming token output, structured tool calling, state management, and MoE synergy for production-grade AI agents.
Read more →
