Entropy Coding
A class of lossless compression methods that assign code lengths according to the probability distribution of symbols, using short codes for high-probability symbols and long codes for low-probability symbols, and that can in theory approach the information entropy of the data.
Timeline (last 90 days)
By exploiting the skewed weight distribution and pairing it with more efficient entropy coding schemes, there is an opportunity to compress the average number of bits per weight to below 1.58.
Storage compression reduces static memory occupancy, but actual inference still requires decoding weights back into a computable form, and excessive decoding overhead may offset the benefits of bandwidth savings.
There is a structural conflict between the variable-length bitstreams produced by entropy coding and the fixed-length, aligned memory layouts preferred by hardware SIMD instructions.
All Facts (3)
By exploiting the skewed weight distribution and pairing it with more efficient entropy coding schemes, there is an opportunity to compress the average number of bits per weight to below 1.58.
50%UnverifiedStorage compression reduces static memory occupancy, but actual inference still requires decoding weights back into a computable form, and excessive decoding overhead may offset the benefits of bandwidth savings.
50%UnverifiedThere is a structural conflict between the variable-length bitstreams produced by entropy coding and the fixed-length, aligned memory layouts preferred by hardware SIMD instructions.
50%