BitNet b1.58
A ternary-weight large language model proposed by Microsoft Research, in which weights are constrained to the values -1, 0, and +1. The ternary constraint is applied from the training stage, making the model natively suited for extremely low-bit inference, with a theoretical storage requirement of approximately 1.58 bits per weight.
Core Facts
Timeline (last 90 days)
BitNet b1.58 restricts weights to the set {-1, 0, +1}, which is highly consistent with the approach taken by Bonsai 2.
The theoretical foundations of ternary quantization can be traced back to 1-bit/ternary network research proposed by Microsoft Research in 2016, and were recently brought back into the spotlight by Meta's BitNet series and BitNet b1.58.
Papers on the BitNet b1.58 series published by Microsoft Research show that a 1.58-bit model with the same parameter count can achieve perplexity close to that of a full-precision model
BitNet b1.58 was proposed by Microsoft Research in 2024.
BitNet b1.58, proposed by Microsoft, restricts each weight to one of three values {-1, 0, 1}, replacing nearly all multiplication operations with addition operations.
BitNet b1.58采用三值量化，每个权重取-1、0或+1三个值，即log₂(3)≈1.58 bit
1-bit量化研究可追溯至微软研究院2024年发布的BitNet论文，提出用三值权重（-1, 0, 1）替代全精度浮点数，BitNet b1.58进一步证明1.58-bit在从头训练场景下的可行性
研究表明当模型参数规模达到3B以上时，1.58-bit模型在多项基准测试中表现可接近甚至匹配同等规模的全精度模型
BitNet b1.58采用三值量化，每个权重取-1、0或+1三个值，约1.58 bit（log₂3）
A系列芯片矩阵运算单元专为低比特整数运算优化，1-bit或三元运算可用加减法代替浮点乘法
1 more timeline events
All Facts (11)
Microsoft Research first proposed BitNet in 2023 and introduced the evolved version BitNet b1.58 in 2024.
65%UnverifiedBitNet b1.58 restricts weights to the set {-1, 0, +1}, which is highly consistent with the approach taken by Bonsai 2.
50%UnverifiedThe theoretical foundations of ternary quantization can be traced back to 1-bit/ternary network research proposed by Microsoft Research in 2016, and were recently brought back into the spotlight by Meta's BitNet series and BitNet b1.58.
50%UnverifiedPapers on the BitNet b1.58 series published by Microsoft Research show that a 1.58-bit model with the same parameter count can achieve perplexity close to that of a full-precision model
50%UnverifiedBitNet b1.58 was proposed by Microsoft Research in 2024.
50%UnverifiedBitNet b1.58, proposed by Microsoft, restricts each weight to one of three values {-1, 0, 1}, replacing nearly all multiplication operations with addition operations.
50%UnverifiedBitNet b1.58采用三值量化，每个权重取-1、0或+1三个值，即log₂(3)≈1.58 bit
50%Unverified1-bit量化研究可追溯至微软研究院2024年发布的BitNet论文，提出用三值权重（-1, 0, 1）替代全精度浮点数，BitNet b1.58进一步证明1.58-bit在从头训练场景下的可行性
50%UnverifiedBitNet b1.58采用三值量化，每个权重取-1、0或+1三个值，约1.58 bit（log₂3）
50%Unverified研究表明当模型参数规模达到3B以上时，1.58-bit模型在多项基准测试中表现可接近甚至匹配同等规模的全精度模型
50%UnverifiedA系列芯片矩阵运算单元专为低比特整数运算优化，1-bit或三元运算可用加减法代替浮点乘法
50%