Unverified50% confidenceFactExact time
FP4-Local uses quantization-aware distillation (QAD): a BF16 teacher model constrains the student model, the student's routed expert weights are quantized to NVFP4 during the forward pass, and gradient updates go to the underlying BF16 weights.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedModels are often trained with FP32 or mixed precision, and are usually converted to FP16 when releasing inference versions to halve the size.68% similarVerifiedQuantization-Aware Training (QAT) lets the model retain a full-precision copy of weights, practicing rounding operations in each forward pass to learn weight distributions under extreme compression.66% similarUnverifiedThe research team fine-tuned the Qwen3.5 series of small language models on filtered and rebalanced teacher-generated supervised data.65% similarVerifiedQLoRA quantizes the base model to 4-bit precision for storage, while applying LoRA adapters on top of the quantized model for training.65% similarUnverifiedLow-precision training and inference (FP16, FP8, FP4) are key to the current ability of large models to scale to trillion parameters.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064225API
curl https://kongchang.com/api/v1/knowledge/claims/1064225MCP
get_claim(id=1064225)