Unverified50% confidenceFactExact time
Both ZIP-SR and ZE-EDEN use 4-bit NormalFloat (NF4) for the first moment and apply targeted stochastic rounding to the first moment of the LM-head during the last 10% of training.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
Unverified实践中所谓的1-bit GGUF通常采用混合方案：embedding层和lm_head等关键组件保留较高精度，中间transformer层使用极低比特，整体平均比特率可能在1.5-2.0 bit/参数之间68% similarUnverifiedIn the experiment, the Qwen3.5-0.8B backbone and approximately 51 billion parameter PLE memory module were frozen, with only R=1 readers deployed at decoder layers 3 and 9 being trained.61% similarUnverified微软DeepSpeed提出的ZeRO技术分为三个阶段：Stage-1分片优化器状态可将内存需求降低约4倍，Stage-2额外分片梯度，Stage-3进一步分片模型参数61% similarUnverifiedLoRA is currently the most mainstream method in parameter-efficient fine-tuning (PEFT); its core approach is to insert a pair of low-rank matrices (typically with a rank of 4 to 64) alongside the weight matrices of Transformer layers, training only these two small matrices.61% similarUnverifiedgg-friggin-ez is powered under the hood by System 1 models, including Jev and Laya from TypeSafe AI.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013232API
curl https://kongchang.com/api/v1/knowledge/claims/1013232MCP
get_claim(id=1013232)