Qwen3-30B-A3B
Qwen3-30B-A3B is a Mixture-of-Experts large language model developed by Alibaba. It has a total of 30 billion parameters but activates only approximately 3 billion parameters per inference step, balancing inference speed with model capability through sparse expert activation.
Core Facts
Timeline (last 90 days)
Running Qwen3-30B-A3B (18.6GB) on a 16GB Tesla T4, using the MoE cache raised decoding speed from the default 22.6 tokens/s to 43.2 tokens/s.
For Qwen3-30B-A3B Q4_K_M with --moe-cache-mib set to 10000 MiB, each of the two 24-layer groups receives 1834 slots, accounting for 59.7% of total experts.
Compared with Qwen3-30B-A3B, Hy-Memory-30B-A3B improves by an average of 11 points, and still scores 56.89 on the EverMemOS framework, which it never saw in training.
Using QLoRA on A100 and RTX PRO 6000, training speed for Qwen3.5-35B-A3B can be improved by up to 4.1×, and for Qwen3-30B-A3B by up to 3.3×.
The off-policy distillation experiment uses Qwen3 0.6B as the student model and Qwen3 4B as the teacher model.
All Facts (6)
The off-policy distillation experiment uses Qwen3 0.6B as the student model and Qwen3 4B as the teacher model.
75%VerifiedQwen3.6 35B-A3B是混合专家架构（MoE），总参数量35B，每次推理实际激活参数量约3B
70%UnverifiedRunning Qwen3-30B-A3B (18.6GB) on a 16GB Tesla T4, using the MoE cache raised decoding speed from the default 22.6 tokens/s to 43.2 tokens/s.
50%UnverifiedFor Qwen3-30B-A3B Q4_K_M with --moe-cache-mib set to 10000 MiB, each of the two 24-layer groups receives 1834 slots, accounting for 59.7% of total experts.
50%UnverifiedCompared with Qwen3-30B-A3B, Hy-Memory-30B-A3B improves by an average of 11 points, and still scores 56.89 on the EverMemOS framework, which it never saw in training.
50%UnverifiedUsing QLoRA on A100 and RTX PRO 6000, training speed for Qwen3.5-35B-A3B can be improved by up to 4.1×, and for Qwen3-30B-A3B by up to 3.3×.
50%