#GPU throughput

391 related articles

Jul 28, 2026·3 min

Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention

Deep dive into Kimi Delta Attention (KDA): from standard Softmax attention's quadratic bottleneck through linear attention, Delta Rule, and gated decay mechanisms — the complete evolution explained.

Jul 28, 2026·3 min

ncnn Vulkan Cross-Platform Edge Inference in Practice: Solving the GPU Fragmentation Deployment Challenge

Facing GPU fragmentation on edge devices, the PostSlate team used ncnn's Vulkan backend for cross-platform ML inference, achieving 10× speedup on RTX 4070 with half the model size and zero runtime installation.

Jul 28, 2026·3 min

Gemini Batch API Major Upgrade: 80% Latency Reduction, Success Rate Exceeds 99.998%

Google's Gemini Batch API completes major infrastructure upgrade with 80% lower p95 latency, 99.998%+ success rate, 98% fewer expirations, and new partial batch support for large-scale AI inference.

Jul 28, 2026·3 min

Kimi K3 Reaches 2.5T Parameters, MiniMax M3 Pro Brewing Simultaneously: China's LLM Arms Race Escalates

Kimi K3 adopts new architecture with 2.5T parameters and 1M token context. MiniMax M3 Pro targets 2.7-3T. OpenAI surpasses 7M users, HY-OCR 1.5 achieves SOTA, Amap launches world model.

Jul 28, 2026·5 min

July 24 AI Daily Brief: Flux 3 Multimodal Release, Kimi K3 Lags in Testing, Qwen Tops TTS Rankings

July 24 AI news: Black Forest Labs launches Flux 3 multimodal model, Kimi K3 lags in US-UK gov tests, Alibaba Qwen tops TTS rankings, Etched raises $300M, AMD unveils MI430X.

Jul 28, 2026·2 min

Claude Paid Subscription Down for Over a Week with No Response: The Pain Points of AI Service Support

Claude AI paid subscription down for over a week with no support response, exposing systemic gaps in AI service customer support. Analysis of impact, industry shortcomings, and user strategies.

Jul 28, 2026·4 min

Cursor Start India Plan at ₹649/Month: Decoding the Regional Pricing Strategy for AI Coding Tools

Cursor launches its Start plan for India at ₹649/month (~$7.70), featuring Grok 4.5 and Composer Agent. A deep dive into the PPP pricing strategy and its market implications.

Jul 28, 2026·4 min

Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop

Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.

Jul 28, 2026·4 min

Laguna S 2.1 Deep Dive: Hands-On Local Deployment of a 118B Parameter Coding Model

In-depth review of Poolside's Laguna S 2.1 open-source coding model: MoE architecture, RL training, DGX Spark local deployment, and real-world agentic coding tests with 8B active parameters.

Jul 28, 2026·3 min

Replacing spaCy's Sentencizer with yasbd: Accuracy Jumps from 55% to 98.9%

spaCy's default Sentencizer achieves only 55.4% accuracy on edge cases, while open-source library yasbd reaches 98.9%. Analysis of limitations and integration code examples.

Jul 28, 2026·3 min

A 9B Model Beats Frontier LLMs After Just $500 in RL Fine-Tuning

Just $500 in RL fine-tuning enables a 9B open-source model to outperform frontier LLMs on catalog review tasks. Analysis of when small-model RL works and its enterprise implications.

Jul 28, 2026·3 min

Apple Silicon LLM Inference Speed Benchmarks: A Complete Guide to Local Large Model Performance on Mac

In-depth analysis of Apple Silicon local LLM inference speed benchmarks covering M-series memory bandwidth, model quantization, MLX framework optimization, and Mac configuration guidance.

Jul 28, 2026·3 min

The Real Daily Life of MLE and MLOps Engineers: Key Differences Between Three Roles and Career Paths

An in-depth look at the real daily work of data scientists, MLEs, and MLOps engineers — covering responsibilities, essential tools, and career paths to help you find your direction in AI.

Jul 28, 2026·2 min

Kimi K3 Launches on Telnyx Inference API: A New Path for Chinese LLMs Going Global

Moonshot AI's Kimi K3 is now available on Telnyx Inference API. Explore how Chinese LLMs are entering global developer ecosystems through third-party inference platforms.

Jul 28, 2026·3 min

AMD CDNA5 Architecture Deep Dive: Technical Evolution and the AI Computing Competition Landscape

Deep analysis of AMD's CDNA5 architecture covering Chiplet packaging upgrades, HBM memory evolution, and low-precision compute optimization, examining how AMD challenges NVIDIA's AI chip dominance.

Jul 27, 2026·3 min

Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide

Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.

Jul 27, 2026·2 min

DeepSeek V4 Official Release Imminent: What 1.6 Trillion Parameters Really Means

DeepSeek V4 official release is imminent. V4 Pro features 1.6 trillion parameters with million-character context. Explore the D-SPARK acceleration framework, dynamic pricing, and V4's leap to production-grade AI.

Jul 27, 2026·3 min

DeepSeek-OCR Deployment & Fine-Tuning in Practice: From vLLM Inference to Unsloth Fine-Tuning

Complete guide to DeepSeek-OCR from vLLM inference deployment and Unsloth model loading to fine-tuning, covering cloud server setup, GPU selection, and code examples — all on a single 4090 GPU.

Jul 27, 2026·1 min

Radxa Cubie A7Z Real-World Test: 330ms End-to-End Object Detection Latency Under Full NPU Load

Deep analysis of Radxa Cubie A7Z achieving 330ms glass-to-glass object detection latency under full NPU load, covering latency breakdown, practical use cases, and edge AI optimization strategies.

Jul 27, 2026·2 min

Google Flash and Flash-Lite Models Upgraded and Live: A New Lightweight, High-Efficiency AI Option

Google rolls out upgraded 3.6 Flash and 3.5 Flash-Lite models. Learn about the Flash series' positioning, upgrade highlights, and value for developers.