Ternary Quantization
An extreme quantization method that constrains neural network weights to three discrete values: -1, 0, and +1, combined with a shared scaling factor to preserve weight magnitude. This approach yields approximately 1.58–1.7 bits per weight and serves as a core technique in research such as BitNet.
Timeline (last 90 days)
Moondream forces each weight in the Parakeet encoder into one of three values—−1, 0, or +1 (ternary quantization), requiring approximately 1.58 bits of storage per weight.
Ternary quantization is applied only to the encoder and not the decoder, which is one of the key reasons for the small accuracy loss.
Bonsai uses ternary quantization, compressing each weight to one of three values—-1, 0, or +1—with a shared scaling factor per 128 weights, requiring approximately 1.7 bits per weight, roughly 9 times smaller than 16-bit full precision.
Ternary quantization restricts weights to three values — -1, 0, and +1 — enabling addition and subtraction to replace multiplication in order to reduce hardware computational load, and is related to Microsoft's BitNet series of research.
Ternary quantization theoretically requires only approximately 1.58 bits (log₂3) per weight for representation.
All Facts (5)
Moondream forces each weight in the Parakeet encoder into one of three values—−1, 0, or +1 (ternary quantization), requiring approximately 1.58 bits of storage per weight.
50%UnverifiedTernary quantization is applied only to the encoder and not the decoder, which is one of the key reasons for the small accuracy loss.
50%UnverifiedBonsai uses ternary quantization, compressing each weight to one of three values—-1, 0, or +1—with a shared scaling factor per 128 weights, requiring approximately 1.7 bits per weight, roughly 9 times smaller than 16-bit full precision.
50%UnverifiedTernary quantization restricts weights to three values — -1, 0, and +1 — enabling addition and subtraction to replace multiplication in order to reduce hardware computational load, and is related to Microsoft's BitNet series of research.
50%UnverifiedTernary quantization theoretically requires only approximately 1.58 bits (log₂3) per weight for representation.
50%