#sparse gradients

57 related articles

Jul 12, 2026·7 min

Training Machine Learning Models from Scratch: A Complete Beginner's Guide

A complete walkthrough of training machine learning models from scratch—covering problem definition, data preprocessing, algorithm selection, hyperparameter tuning, and evaluation, with tool recommendations for beginners.

Jul 12, 2026·6 min

DeepSeek Builds Its Own AI Chips: The Strategic Logic and Challenges Behind Compute Independence

DeepSeek is entering AI chip development, targeting compute autonomy. This article analyzes its motivations, software-hardware synergy, chip R&D challenges, and impact on China's AI vertical integration.

Jul 11, 2026·7 min

Practical RL Compute Optimization: Core Strategies to Boost CPU and GPU Utilization

An in-depth look at why CPU and GPU utilization is low in RL training, covering vectorized environment parallelism, distributed Actor-Learner architectures, GPU-side simulation (Isaac Gym/Brax), and Ray RLlib practice.

Jul 11, 2026·3 min

Reward Shaping and RL Debugging in Practice: A Systematic Guide from Oscillation to Convergence

Using an FPV drone RL project as a case study, this guide covers reward shaping principles, Bang-Bang control hacking, module isolation, single-variable debugging, and behavior visualization to solve common RL training issues.

Jul 10, 2026·6 min

Tencent Hunyuan HY3 Official Version Open-Sourced: The Strongest Under 300B, at 1 Yuan per Million Tokens

Tencent Hunyuan HY3 official version is open-sourced under Apache 2.0, priced as low as 1 yuan per million input tokens, with major gains in agents, reasoning, coding, and long context. On the same day, Meituan open-sourced its trillion-parameter LongCat 2.0.

Jul 10, 2026·4 min

DeepSeek-V4 Deep Dive: Million-Token Context Architecture Breakthrough and Core Innovations

Deep dive into DeepSeek-V4: 1.6T-parameter MoE, CSA+HCA hybrid attention, MHC & MUON optimizer. Inference FLOPs drop to 27% of V3.2, redefining open-source LLM SOTA.

Jul 10, 2026·7 min

Complete Guide to Training Small Models on Kaggle's Free Compute: GPU Quotas and Practical Tips

In-depth guide to Kaggle's free-tier compute: P100/T4 GPU with 30 hours/week quota, 12-hour sessions, suitable models like CNN and BERT fine-tuning, plus tips like mixed precision and checkpointing to start deep learning at zero cost.

Jul 9, 2026·7 min

The Agent Engineer Transition Roadmap: A Deep Dive into Three Core Skill Systems

Want to become an Agent engineer? This article systematically covers three core skill tracks—LLM fundamentals, LangChain architecture development, and enterprise deployment—to help you avoid detours.

Jul 8, 2026·7 min

Metacognitive Feedback: Using Reinforcement Learning to Help LLMs Accurately Express Uncertainty

LLMs are often overconfident and prone to hallucination. How can AI learn to say "I'm not sure"? This article explains the reinforcement learning approach with metacognitive feedback and how calibrating confidence boosts LLM trustworthiness.

Jul 8, 2026·3 min

Unsloth v0.1.45-beta Released: 2x Faster LLM Fine-Tuning with 70% Less VRAM

Unsloth v0.1.45-beta (PyPI: 2026.6.2) delivers 2x faster LLM fine-tuning and up to 70% VRAM reduction. Now at 67.9k GitHub stars, upgrade via pip install.

Jul 7, 2026·7 min

Dispersion Loss: A Key Technique for Solving Embedding Condensation in Small Language Models

Embedding condensation is a hidden bottleneck in small language model training. Dispersion Loss combats this by enforcing representation spread during training at zero inference cost.

Jul 7, 2026·4 min

Why Large Language Models Are Starting to 'Invent Words': The Nature of LLM Language Boundaries

Top LLMs are pushing beyond existing human vocabulary, producing neologisms and expressive distortion. This article analyzes the tension between LLM high-dimensional semantic spaces and natural language symbol systems.

Jun 13, 2026·3 min

Biological Neurons vs. Artificial Neurons: How Big Is the Computational Gap?

Research shows biological neurons far outperform classical artificial neurons. Explore dendritic computing, temporal coding, and what this means for next-gen AI architecture design.

Python Machine Learning in Practice: A Complete Guide from Algorithm Derivation to Code Implementation

Tutorials

May 21, 2026·2 min

Python Machine Learning in Practice: A Complete Guide from Algorithm Derivation to Code Implementation

A systematic Python ML course using a 3-stage framework: algorithm derivation, code implementation, and experimental analysis. Covers logistic regression, decision trees, and ensemble learning.

NCCL Multi-GPU Communication Optimization in Practice: A Guide to Scaling HPC High-Performance Computing

Tutorials

May 19, 2026·2 min

NCCL Multi-GPU Communication Optimization in Practice: A Guide to Scaling HPC High-Performance Computing

Deep dive into NVIDIA NCCL multi-GPU communication library principles and optimization strategies, covering AllReduce, NVLink, and GPUDirect RDMA to help HPC and AI developers master scaling from single-node to massive clusters.

98 Expert Agents Working in Concert: A Deep Dive into an Open-Source Multi-Agent Orchestration Framework

Deep Dives

May 13, 2026·3 min

98 Expert Agents Working in Concert: A Deep Dive into an Open-Source Multi-Agent Orchestration Framework

Deep dive into a trending open-source multi-agent framework with 98 expert agents, swarm orchestration, HNSW vector memory, autonomous learning, and Agent Federation for distributed collaboration.

Unsloth: 80% VRAM Reduction — The Best Open-Source Tool for Local LLM Fine-Tuning

Product Reviews

May 13, 2026·2 min

Unsloth: 80% VRAM Reduction — The Best Open-Source Tool for Local LLM Fine-Tuning

Unsloth is an open-source LLM training tool with 63K+ GitHub stars, supporting Gemma 4, Qwen 3, DeepSeek. Reduces VRAM by 50–80%, enabling RTX 4090 to fine-tune 7B models with a no-code Web UI.