#continuous batching

96 related articles

Jul 11, 2026·2 min

MOSS-Transcribe-Diarize: A 0.9B End-to-End Multi-Speaker Speech Transcription Model

MosiAI open-sources MOSS-Transcribe-Diarize-0.9B: unified speech transcription and speaker diarization, 128K context for 90-min audio, hotword boosting, SGLang Day-0 support, edge-deployable.

Jul 11, 2026·5 min

Ollama Raises $65M Series B: 85% of Fortune 500 Have Already Deployed Local LLMs

Local LLM tool Ollama closes a $65M Series B, bringing total funding to $88M. With 9M developers and 85% of Fortune 500 having deployed internally, this deep dive explores why enterprises embrace local LLMs: compliance, Agent cost savings, and open-source ecosystem.

Jul 11, 2026·3 min

Connecting Codex to Ollama: A Complete Guide to Deploying AI Large Models Locally

A step-by-step guide to combining Codex with Ollama to deploy open-source AI large models locally. Private data, no subscription, offline operation, no VPN needed. Includes hardware selection and setup.

Jul 11, 2026·4 min

GSAP in Practice: Build Stunning Landing Page Animations from Scratch

Master GSAP animations in practice: ScrollTrigger, SplitText, Timeline orchestration, and frame-rate-consistent infinite scroll — with full code examples. Now completely free.

Jul 11, 2026·6 min

Cursor Meets Grok 4.5: A Hands-On Test of an AI Coding Assistant That's 5x Faster with Cleaner Output

A developer deeply tests Grok 4.5 High Fast in Cursor, finding it rivals Claude Opus in quality but runs 5x faster with cleaner, filler-free output. Full hands-on review and analysis.

Jul 10, 2026·5 min

Real-World Comparison of 6 Open-Source LLMs for On-Premise Deployment: A Selection Guide to Avoid Pitfalls

A hands-on comparison of 6 open-source LLMs (DeepSeek, Qwen3, Zhipu GLM, Kimi K2, MiniMax M3, Tencent Hunyuan 3) for on-premise deployment—covering hardware cost, inference efficiency, and deployment difficulty.

Jul 10, 2026·5 min

DeepSeek Boosts Inference Speed by 85%: How Speculative Decoding Is Reshaping the AI Speed Race

DeepSeek and Peking University's DS Spark paper boosts AI inference speed by up to 85% via confidence scheduling and semi-autoregressive speculative decoding — no model or GPU changes.

Jul 10, 2026·6 min

AI's Second Half: The Decisive Battle Shifts from Models to Infrastructure

Model capabilities are converging, making inference cost and scalability the new focus of AI competition. A deep analysis of AI infrastructure's core layers.

Jul 10, 2026·5 min

TensorRT Multi-Device Inference: Engineering Practices for Breaking Through Single-GPU Memory Bottlenecks

NVIDIA TensorRT now supports multi-device inference via pipeline and tensor parallelism, distributing large models across multiple GPUs to break through single-card memory limits.

Jul 10, 2026·3 min

Muse Spark 1.1 Released: Competing Through Ultra-Low Cost in a Crowded AI Tool Market

Muse Spark 1.1 launches with an ultra-low cost focus. We break down the pricing strategy, technical approaches behind it, and its real value for developers and small teams.

Jul 10, 2026·7 min

Getting Started with LangChain 1.3: Core Concepts of LLMs and Agents

An in-depth breakdown of LangChain 1.3's core concepts, covering the three major limitations of LLMs, Agent architecture, memory management, and a complete learning path. Master LangChain and LangGraph to quickly build AI development skills.

Jul 9, 2026·5 min

Is AI Inference Really Profitable? Deconstructing the Cost Economics of Inference vs. Training

Are AI companies really losing money? This article deconstructs the fundamental differences between training and inference costs, revealing why pure AI inference services have structural profitability.

Jul 9, 2026·6 min

Xiaomi XiaoAI 10.1-inch Smart Control Panel: Is the AI LLM Integration Worth It?

Xiaomi XiaoAI 10.1-inch Smart Control Panel features AI LLM Q&A, WeChat calling, and whole-home Mi IoT control. Priced at 839 yuan, ~679 yuan after national subsidy. An in-depth review of AI capabilities, screen experience, and smart home integration.

Jul 9, 2026·7 min

LangChain Hands-On Guide: From LLM Calls to Agent Development

Master LangChain from scratch: the three limitations of LLMs, init_chat_model unified interface config, the Message type system, and the path from LLM calls to Agent development.

Jul 8, 2026·5 min

Claude Cowork Arrives on Mobile and Web, with Max Users Getting Early Access

Anthropic's AI collaboration platform Claude Cowork officially expands to mobile and web, breaking free from desktop-only limits. Max subscribers get early access, with other plans rolling out over the coming weeks.

Jul 8, 2026·7 min

LangChain.js + Nuxt.js: An AI Full-Stack Advancement Path for Frontend Engineers

Frontend hiring now treats AI capabilities as a core assessment, covering RAG knowledge bases, AI Agent development, and LangChain.js engineering. Learn how LangChain.js + Nuxt.js helps frontend developers build memory- and retrieval-capable AI full-stack apps.

Jul 7, 2026·6 min

Reverse-Engineering ChatGPT's Source Selection: What Network Traffic Analysis Reveals About AI Citations

By capturing and analyzing ChatGPT's real network traffic during web retrieval, this article reveals the underlying logic of AI source selection—covering the layered mechanisms of retrieval recall and content filtering, domain preference patterns, and practical insights for content creators and SEO professionals.

Jul 7, 2026·6 min

GLM 5.2 Arrives: Industry Upheaval on the Eve of AI Margin Collapse

The rise of Zhipu's GLM 5.2 is accelerating the democratization of LLM capabilities. This article analyzes the commoditization of foundation models, the logic behind margin collapse, and the opportunities and challenges facing application-layer and foundation model firms.

Jul 7, 2026·4 min

NVFP4 Quantization in Practice: A Guide to Compressing Large Models with NVIDIA Model Optimizer

Deep dive into NVFP4 quantization: using NVIDIA Model Optimizer to compress Nemotron 3 Ultra to FP4 checkpoints, reducing memory by 75% and boosting inference throughput on Blackwell GPUs.

Jul 7, 2026·5 min

NVIDIA AI-Q Blueprint × OCI: A Practical Guide to Production Deployment of Enterprise AI Agents

Deep dive into NVIDIA AI-Q Blueprint production deployment on Oracle Cloud Infrastructure, covering NIM microservices, RAG architecture, multi-agent orchestration, and OCI GPU selection for enterprise AI agents.