#tensor parallelism

122 related articles

Jul 12, 2026·5 min

The Five Unsolved Challenges of Production MLOps: A Deep Dive into GPU Scheduling, Fault-Tolerant Training, and Model Observability

A deep dive into the five genuinely tough challenges of production MLOps: fault-tolerant training on Spot instances, cross-team GPU scheduling, data reproducibility, model observability, and inference cost optimization.

Mesh LLM: A Practical Exploration of B…

Jul 12, 2026·3 min

Mesh LLM: A Practical Exploration of Building a P2P Distributed AI Inference Network with iroh

Mesh LLM leverages the Rust P2P framework iroh to integrate compute from scattered nodes, exploring a viable path for decentralized LLM inference. This article analyzes its architecture, challenges, and prospects.

Jul 12, 2026·3 min

SlickToken: A Deep Dive into GPU Fleet and Agentic Workflow Planning

SlickToken is a GPU fleet and agentic workflow planning tool for AI teams, supporting offline simulation, load testing, and capacity planning—no internet required to protect enterprise data.

Jul 12, 2026·3 min

Benchmarking LLMs on Smartphones: A Deep Dive into the MELTing Point Benchmark

The MELTing Point paper is the first to evaluate mobile LLM performance in real user scenarios, covering iPhone, Samsung, Pixel and more, testing TinyLlama, Mistral-7B and others—revealing GPU inference gains, 47°C heat warnings, and prefill-decode disaggregation.

Jul 12, 2026·2 min

Decoding the AI Full-Stack Approach: Google's Vertical Integration Strategy from Chips to Applications

Deep analysis of Google's AI full-stack strategy: from custom TPU chips and system software frameworks to Gemini models and applications, examining how vertical integration delivers performance, cost, and autonomy advantages.

Jul 12, 2026·3 min

DeepSeek-V4 Flash RL Training Successfully Ported to AMD MI355X GPU: A Complete Breakdown

The Miles team and AMD announce the full port of DeepSeek-V4 Flash RL training to AMD Instinct MI355X GPUs on ROCm, boosting AIME pass@1 from 0.39 to 0.49—a milestone for compute ecosystem diversity.

Jul 11, 2026·7 min

Practical RL Compute Optimization: Core Strategies to Boost CPU and GPU Utilization

An in-depth look at why CPU and GPU utilization is low in RL training, covering vectorized environment parallelism, distributed Actor-Learner architectures, GPU-side simulation (Isaac Gym/Brax), and Ray RLlib practice.

Jul 11, 2026·6 min

A Deep Dive into the vLLM Inference Framework: Core Principles of Throughput Optimization and Interview Guide

An in-depth analysis of the vLLM inference framework's core principles: from the meaning of throughput (tokens/s), to the bottlenecks of autoregressive generation, to KV Cache, PagedAttention, and continuous batching.

Jul 11, 2026·6 min

Grok 4.5 Hands-On: A Faster, Cheaper Coding Dark Horse — Can It Replace Opus and GPT?

In-depth Grok 4.5 hands-on review: priced at a fraction of Opus 4.8, twice the token efficiency of peers, and coding ability in the top tier. A real-project breakdown of its strengths, highlights, and shortcomings.

Jul 11, 2026·6 min

GPT Live Full-Duplex, Grok 4.5 Coding, Seedream 5.0 Images: AI Accelerates Across Three Tracks Simultaneously

OpenAI's GPT Live full-duplex voice model, Grok 4.5 coding model with Cursor, and ByteDance's Seedream 5.0 Pro image generation launched together. A deep dive into three AI releases.

Jul 11, 2026·2 min

AI Agents for SGLang Kernel Optimization: Real-World Performance Gains and Engineering Methodology

SGLang's team converted expert knowledge into agent skills, achieving 71.4% throughput gains, TTFT reduced from 456ms to 168ms. A deep dive into agent-assisted kernel optimization methodology.

Jul 11, 2026·4 min

Demystifying Unified Memory Architecture: How Mini PCs Run 70B Large Models

Why can a mini PC with unified memory run a 70B model while an RTX 4090 can't? A deep dive into the VRAM wall and unified memory architecture for smarter local AI hardware choices.

Jul 10, 2026·5 min

Real-World Comparison of 6 Open-Source LLMs for On-Premise Deployment: A Selection Guide to Avoid Pitfalls

A hands-on comparison of 6 open-source LLMs (DeepSeek, Qwen3, Zhipu GLM, Kimi K2, MiniMax M3, Tencent Hunyuan 3) for on-premise deployment—covering hardware cost, inference efficiency, and deployment difficulty.

Jul 10, 2026·5 min

TensorRT Multi-Device Inference: Engineering Practices for Breaking Through Single-GPU Memory Bottlenecks

NVIDIA TensorRT now supports multi-device inference via pipeline and tensor parallelism, distributing large models across multiple GPUs to break through single-card memory limits.

Jul 10, 2026·7 min

Complete Guide to Training Small Models on Kaggle's Free Compute: GPU Quotas and Practical Tips

In-depth guide to Kaggle's free-tier compute: P100/T4 GPU with 30 hours/week quota, 12-hour sessions, suitable models like CNN and BERT fine-tuning, plus tips like mixed precision and checkpointing to start deep learning at zero cost.

Jul 9, 2026·4 min

AI Weekly: GPT-5.6 Launches in Three Tiers, Open-Source Inference Frameworks and Robot Safety Break Through Across the Board

This week in AI: OpenAI launches GPT-5.6 in three tiers (Sol/Terra/Luna) hitting 91.9% on coding benchmarks; DeepSeek and PKU open-source DSpark for 85% faster inference; Prime Intellect trains trillion-param models on just 28 H200s; Anthropic Claude enters Slack.

Jul 9, 2026·4 min

Unsloth v0.1.463-beta: Fixes Studio Crash and llama-server Permission Issue

Unsloth v0.1.463-beta fixes a Studio crash caused by access-denied errors during llama-server service discovery. Improves stability for multi-user servers and Windows environments.

Jul 9, 2026·7 min

LangChain Hands-On Guide: From LLM Calls to Agent Development

Master LangChain from scratch: the three limitations of LLMs, init_chat_model unified interface config, the Message type system, and the path from LLM calls to Agent development.

Jul 9, 2026·7 min

Getting Started with OpenAI LLM Development: A Practical Guide to Model Selection, Token Pricing, and API Usage

A systematic guide to the three cores of OpenAI LLM app development: GPT-4/GPT-3.5 model selection, token billing and cost-saving tips, and practical use of the Models, Completion, and Chat Completion APIs.

Jul 8, 2026·7 min

Learning Python from Scratch: A Complete Guide to the Three-Stage Learning Path

Want to learn Python from scratch but don't know where to begin? This article breaks down three stages—basic syntax, advanced mastery, and hands-on practice—with real projects in crawling, automation, and data analysis to help you build programming thinking.