Mamba2-8B
An 8-billion-parameter sequence modeling model based on the Mamba2 state space architecture. Its core advantages are linear time complexity and a constant-size state representation, so the inference cache does not grow with context length.
Timeline (last 90 days)
A research series on Mamba2-8B model compression has appeared in the Reddit community, exploring two strategies: a 27.18 MiB recurrent cache and 3.141 GB full model data storage.
27.18 MiB is approximately equal to 22.7 million bytes; if stored at 8 bits, this corresponds to about 22.7 million parameters, accounting for only 0.28% of an 8B model.
The size of the recurrent cache is only about 0.86% of the full model.
The 27.18 MiB cache size makes it possible for Mamba2-8B to run on high-end mobile devices or even embedded systems.
The community still needs to pay attention to the quality loss of compressed models, including whether the 27.18 MiB cache can preserve the inference quality of the original model, and the performance difference between the 3.141 GB storage version and the uncompressed version.
All Facts (5)
27.18 MiB is approximately equal to 22.7 million bytes; if stored at 8 bits, this corresponds to about 22.7 million parameters, accounting for only 0.28% of an 8B model.
50%UnverifiedThe size of the recurrent cache is only about 0.86% of the full model.
50%UnverifiedThe 27.18 MiB cache size makes it possible for Mamba2-8B to run on high-end mobile devices or even embedded systems.
50%UnverifiedThe community still needs to pay attention to the quality loss of compressed models, including whether the 27.18 MiB cache can preserve the inference quality of the original model, and the performance difference between the 3.141 GB storage version and the uncompressed version.
50%UnverifiedA research series on Mamba2-8B model compression has appeared in the Reddit community, exploring two strategies: a 27.18 MiB recurrent cache and 3.141 GB full model data storage.
50%