Qwen3-0.6B
An ultra-small language model in Alibaba's Qwen series, with 0.6 billion parameters. It is designed for edge devices, on-device inference, and low-cost deployment scenarios.
Core Facts
Timeline (last 90 days)
The weights of the small Qwen3-0.6B model are only about 1-2GB.
In a comparison of Qwen3-0.6B in BF16 versus Q4_K_M, Q4_K_M changed the top-1 token at 11.6% of positions, with an average KL divergence of 0.067 and perplexity rising from 7.83 to 8.53.
At the is_prime function definition, the BF16 version has a 97.9% probability of outputting False, whereas Q4_K_M has a 79.1% probability of outputting True.
Layer-by-layer denoising experiments between Q4_K_M and BF16 show KL decreasing smoothly from layer 0 to layer 27, with no single layer carrying all the difference.
At the is_prime token, after writing BF16's residual into Q4_K_M, that token's KL dropped from 0.48 after layer 6 to 0.04 after layer 9; the reverse write reproduced the flip from about layer 12, indicating the decisive difference forms between layers 6 and 12.
Across 12 blocks and 1282 scored positions, when Q4_K_M recovers BF16, attention eliminates 0.030 (95% CI 0.019–0.041) and MLP eliminates 0.036 (0.024–0.048), with the paired difference crossing zero so the two are indistinguishable; when injecting Q4_K_M into BF16, MLP adds more (0.014, 0.005 to 0.023).
Qwen2.5-1.5B scores 0.729 on GSM8K, 18.2 percentage points lower than Gemma 4 E2B; Qwen3-0.6B scores 0.599, 31.2 percentage points lower; SmolLM2-1.7B scores 0.461, 45.0 percentage points lower.
Candidate small models include SmolLM2 (135M and 360M), Gemma 3 270M, LFM2.5-350M, Qwen2.5-0.5B, and Qwen3-0.6B, all of which have ONNX browser builds.
The ByteDance team's control experiments trained from scratch using the Qwen3-0.6B architecture show that all models using chunked compression exhibit periodic accuracy fluctuations, with the fluctuation period strictly equal to the compression step; the fluctuations still exist after removing RoPE or switching to evenly distributed gating weights; and the uncompressed full-attention baseline is completely normal.
Alibaba's Qwen/Qwen3-0.6B added over 1.1 million downloads in one week, jumping from approximately 29.66 million to 30.76 million, making it the model with the largest download increment this week.
2 more timeline events
All Facts (13)
The Qwen2.5:7B model is approximately 4 GB in size after Q4 quantization.
80%UnverifiedThe weights of the small Qwen3-0.6B model are only about 1-2GB.
50%UnverifiedAcross 12 blocks and 1282 scored positions, when Q4_K_M recovers BF16, attention eliminates 0.030 (95% CI 0.019–0.041) and MLP eliminates 0.036 (0.024–0.048), with the paired difference crossing zero so the two are indistinguishable; when injecting Q4_K_M into BF16, MLP adds more (0.014, 0.005 to 0.023).
50%UnverifiedIn a comparison of Qwen3-0.6B in BF16 versus Q4_K_M, Q4_K_M changed the top-1 token at 11.6% of positions, with an average KL divergence of 0.067 and perplexity rising from 7.83 to 8.53.
50%UnverifiedAt the is_prime function definition, the BF16 version has a 97.9% probability of outputting False, whereas Q4_K_M has a 79.1% probability of outputting True.
50%UnverifiedLayer-by-layer denoising experiments between Q4_K_M and BF16 show KL decreasing smoothly from layer 0 to layer 27, with no single layer carrying all the difference.
50%UnverifiedAt the is_prime token, after writing BF16's residual into Q4_K_M, that token's KL dropped from 0.48 after layer 6 to 0.04 after layer 9; the reverse write reproduced the flip from about layer 12, indicating the decisive difference forms between layers 6 and 12.
50%UnverifiedQwen2.5-1.5B scores 0.729 on GSM8K, 18.2 percentage points lower than Gemma 4 E2B; Qwen3-0.6B scores 0.599, 31.2 percentage points lower; SmolLM2-1.7B scores 0.461, 45.0 percentage points lower.
50%UnverifiedCandidate small models include SmolLM2 (135M and 360M), Gemma 3 270M, LFM2.5-350M, Qwen2.5-0.5B, and Qwen3-0.6B, all of which have ONNX browser builds.
50%UnverifiedThe ByteDance team's control experiments trained from scratch using the Qwen3-0.6B architecture show that all models using chunked compression exhibit periodic accuracy fluctuations, with the fluctuation period strictly equal to the compression step; the fluctuations still exist after removing RoPE or switching to evenly distributed gating weights; and the uncompressed full-attention baseline is completely normal.
50%UnverifiedAlibaba's Qwen/Qwen3-0.6B added over 1.1 million downloads in one week, jumping from approximately 29.66 million to 30.76 million, making it the model with the largest download increment this week.
50%UnverifiedQwen3-0.8B is a small-parameter model that can be trained on a single consumer-grade GPU.
50%UnverifiedNo-padding training reduces Qwen3-0.6B's VRAM usage from 5.45 GB to 1.87 GB, with a per-step speed improvement of 5 to 8 times.
50%