EXL3
A quantized container format for large language models optimized for NVIDIA GPUs. It supports fine-grained weight quantization and is suited for high-performance local inference scenarios.
Timeline (last 90 days)
DeepSeek-V4.1-Flash has 763B parameters, BF16 weights are about 1.5 TB, the official FP4/FP8 checkpoint is 475.2 GiB, and coolbho3k's 3-bit EXL3 package compresses it to 309.3 GiB.
EXL3 is a 3-bit quantization format introduced by ExLlamaV3 and is the successor to EXL2; ExLlamaV3 is an inference library maintained by turboderp; vllm-exl3 is a bridge layer written by Victor Cruz that connects ExLlamaV3's quantization kernels into vLLM's scheduling and PagedAttention system.
EXL3 uses vector and lattice quantization derived from QTIP, retaining more information at the same bit count than scalar rounding.
All Facts (3)
DeepSeek-V4.1-Flash has 763B parameters, BF16 weights are about 1.5 TB, the official FP4/FP8 checkpoint is 475.2 GiB, and coolbho3k's 3-bit EXL3 package compresses it to 309.3 GiB.
50%UnverifiedEXL3 is a 3-bit quantization format introduced by ExLlamaV3 and is the successor to EXL2; ExLlamaV3 is an inference library maintained by turboderp; vllm-exl3 is a bridge layer written by Victor Cruz that connects ExLlamaV3's quantization kernels into vLLM's scheduling and PagedAttention system.
50%UnverifiedEXL3 uses vector and lattice quantization derived from QTIP, retaining more information at the same bit count than scalar rounding.
50%