[KongchangAI]
ConceptQAD

Quantization-Aware Distillation

A compression method that trains low-precision models directly under quantization constraints, embedding quantization operators into the training graph and using the outputs of a full-precision teacher to supervise the low-precision weights.

Timeline (last 90 days)

Oct 17

NVIDIA's FP4 version of Qwen3.8 was trained using quantization-aware distillation (QAD), with a training budget of 3 epochs totaling 500M tokens, comprising 2,500 backbone updates and 1,500 joint fine-tuning updates.

Unverified50%

All Facts (1)

Unverified

NVIDIA's FP4 version of Qwen3.8 was trained using quantization-aware distillation (QAD), with a training budget of 3 epochs totaling 500M tokens, comprising 2,500 backbone updates and 1,500 joint fine-tuning updates.

50%

Source Articles

GLM-5.3-Flash Local Deployment: NVIDIA FP4 Requires Two Blackwell GPUsOct 18Local Deployment of NVIDIA's FP4 Qwen3.8: Blackwell-Exclusive and 55 Environment VariablesOct 17