#continuous batching
85 related articles
·3 min
The Real Daily Life of MLE and MLOps Engineers: Key Differences Between Three Roles and Career Paths
An in-depth look at the real daily work of data scientists, MLEs, and MLOps engineers — covering responsibilities, essential tools, and career paths to help you find your direction in AI.
Read more →
·3 min
DeepSeek-OCR Deployment & Fine-Tuning in Practice: From vLLM Inference to Unsloth Fine-Tuning
Complete guide to DeepSeek-OCR from vLLM inference deployment and Unsloth model loading to fine-tuning, covering cloud server setup, GPU selection, and code examples — all on a single 4090 GPU.
Read more →
·3 min
HuggingHack Update: S3 Storage, Dual-Engine Scheduling, and GGUF Inspection Get Major Upgrades
HuggingHack releases major updates with S3/MinIO storage, Ollama + vLLM dual-engine scheduling, GGUF inspection, and local accounts for enterprise-grade local LLM management.
Read more →
·3 min
DeepSeek-OCR Deployment and Fine-Tuning: A Complete Hands-On Guide
Master the full DeepSeek-OCR deployment and fine-tuning workflow: vLLM inference deployment, efficient Unsloth fine-tuning, dataset preprocessing, LoRA training, validation, and RAG vector database integration.
Read more →
·3 min
DeepSeek Open-Sources DeepSpec: A Full-Stack Speculative Decoding Library with 60–85% Inference Speedup
DeepSeek open-sources DeepSpec, a full-stack speculative decoding library delivering 60–85% inference speedup and up to 661% throughput gains in production. MIT license, 9 checkpoints, 9 benchmarks.
Read more →
·3 min
Is Gemini Getting Worse? A Deep Dive into the 'Model Degradation' Phenomenon
Reddit users report Gemini feeling "worse over time." This deep dive explains the real causes—silent updates, safety over-alignment, and compute throttling—plus how to rationally evaluate AI model performance.
Read more →
Kronos Financial Foundation Model: Usi…
·7 min
Kronos Financial Foundation Model: Using AI to Read the Market Language of Candlesticks
Kronos is the first open-source foundation model treating candlestick data as the "language of financial markets," using an autoregressive Transformer and earning 32K GitHub Stars. A deep dive into its principles, applications, and limits.
Read more →
·3 min
Qwen 3.8 Open Source Incoming, DeepSeek Coding Agent, and the Latest in China's AI Ecosystem
Qwen 3.8 open source incoming, DeepSeek Resonics native coding agent debuts, Kimi pauses new subscriptions amid compute pressure, and Qiyimole shows domestic GPU interconnect solution at WAIC.
Read more →
·7 min
Alibaba Open-Sources 2.4T Qwen Model as Chinese AI Sweeps the Global Competitive Landscape
Alibaba open-sources a 2.4 trillion parameter Qwen model and launches the Qwen Token Plan. Chinese models surge, Kimi K3 tops global rankings, and China's AI is reshaping the global competitive landscape.
Read more →
·3 min
Claude Opus 5 Launch Imminent, Alibaba Qwen Officially Integrated into Apple Intelligence
Claude Opus 5 launches next week; Alibaba Qwen integrates into Apple Intelligence for Chinese users; 27B on-device model compressed to 3.8GB; open-source models narrow gap to closed-source by 3.3%.
Read more →
·2 min
GPT-Live Real-Time Voice Interaction: How "Listen and Speak Simultaneously" Enables Natural Conversation
GPT-Live breaks the "turn-based" limit with millisecond real-time voice interaction. Explore the tech behind simultaneous listen-and-speak, multilingual interpretation demos, and AI voice's shift from tool to conversational partner.
Read more →
Handwritten C/CUDA Inference Engine: P…
·2 min
Handwritten C/CUDA Inference Engine: Pushing Qwen 35B to the Limits on RTX 5090
A deep dive into a handwritten C/CUDA inference engine for Qwen 35B on RTX 5090 (Blackwell), covering quantization, FlashAttention kernels, and memory optimization.
Read more →
·4 min
AMD Ryzen AI Halo Real-World Test: Can a Non-Programmer Use Local AI to Write Code?
A non-programmer tests AMD Ryzen AI Halo by deploying local AI models to tackle a real dev task. After testing Ollama and Qwen3, the verdict: AI amplifies developers, it doesn't replace them.
Read more →
NVIDIA DeepStream 9.1 Multi-Camera 3D …
·2 min
NVIDIA DeepStream 9.1 Multi-Camera 3D Tracking: Principles, Architecture, and Applications
A deep dive into NVIDIA DeepStream 9.1 multi-camera 3D tracking: cross-camera Re-ID, 3D coordinate fusion, GPU-accelerated pipelines, and real-world deployments in retail, warehousing, and security.
Read more →
·2 min
Variable-Length Matrix Classification: Choosing the Right Neural Network Architecture for Unordered Set Data
Why do CNNs and RNNs fail on unordered matrix data? Learn about permutation invariance, Deep Sets, and Set Transformer to pick the right architecture for set-based classification.
Read more →
·2 min
vLLM Inference Framework Deep Dive: A Practical Guide to High-Concurrency DeepSeek Deployment
A deep dive into vLLM's five core features — KV cache management, continuous batching, and distributed inference — plus a full setup guide for deploying DeepSeek on a cloud server.
Read more →
·3 min
New Retrieval Paradigms: Hobbit's Hard Negative Construction and Disco's Collaborative Coverage Index Design
A deep dive into two frontier dense retrieval works: Hobbit uses gradient analysis to automatically construct hard batches; Disco replaces single-document competition with submodular collaborative coverage, reshaping Top-K retrieval.
Read more →
·4 min
Agent Loop in Practice: A Guide to Building Self-Improving Automation with Guardrails
A deep dive into building verifiable, self-evolving Agent automation loops with Claude Code and Codex — covering Loop Contracts, four trigger types, three-phase execution architecture, and Evolve Loops.
Read more →
·3 min
Semantic Caching: The Core Technology and Practical Guide to Cutting AI Costs by 50%
Semantic caching uses vector embeddings to match similar queries and avoid redundant LLM calls, cutting AI app costs by up to 50%. Learn how it works, how to set thresholds, and when to use it.
Read more →
·2 min
Production LLM Deployment Guide: Model Selection, GPU Configuration, and Inference Engine Breakdown
A comprehensive decision framework for production LLM deployment: model selection (open-source vs. API), GPU VRAM configuration, and inference engine comparison including vLLM and TensorRT-LLM.
Read more →