coolbho3k
An open-source developer working on large model quantization who produced a 3-bit EXL3 quantization package for DeepSeek-V4.1-Flash.
Timeline (last 90 days)
DeepSeek-V4.1-Flash has 763B parameters, BF16 weights are about 1.5 TB, the official FP4/FP8 checkpoint is 475.2 GiB, and coolbho3k's 3-bit EXL3 package compresses it to 309.3 GiB.
The working solution relies on a combination: DeepSeek provides the weights, coolbho3k does the 3-bit quantization, turboderp wrote the EXL3 format and the ExLlamaV3 inference library, Victor Cruz wrote the vllm-exl3 bridge layer, Diffbot designed the per-expert split scheme, startup flow and kernels, and InferenceKey did the integration and measurement.
All Facts (2)
DeepSeek-V4.1-Flash has 763B parameters, BF16 weights are about 1.5 TB, the official FP4/FP8 checkpoint is 475.2 GiB, and coolbho3k's 3-bit EXL3 package compresses it to 309.3 GiB.
50%UnverifiedThe working solution relies on a combination: DeepSeek provides the weights, coolbho3k does the 3-bit quantization, turboderp wrote the EXL3 format and the ExLlamaV3 inference library, Victor Cruz wrote the vllm-exl3 bridge layer, Diffbot designed the per-expert split scheme, startup flow and kernels, and InferenceKey did the integration and measurement.
50%