1 Bit Per Parameter! 763B DeepSeek-V4.1-Flash Compressed Edition Released

A New Breakthrough in Model Compression: Squeezing 763B Parameters into 1 Bit

Large model compression technology is seeing a rather imaginative attempt. ISTA Deep Algorithms and Systems Lab released a Coder build of DeepSeek^Ad-V4.1-Flash-1017 on Reddit, using two techniques—GSQ and RCO—to compress the original 763.2B-parameter sparse mixture-of-experts model down to an average of 1 bit per parameter, while retaining half of the experts.

DeepSeek-V4.1-Flash-1017 is itself a sparse MoE model: each layer has 384 routed experts, 6 of which are activated per token, with 763.2B total parameters, of which 551.9B reside in the transformer portion. This Coder release retains 192 experts per layer (i.e., half are pruned), with transformer weight size at 72.1 GB, compared to 138.2 GB for the unpruned IQ2_XXS build.

How Is the 1 Bit Calculated?

The official explanation of the "1 bit per parameter" calculation is explicit: 72.12 GB divided by the original 551.9B transformer parameters gives 1.05 bits, rounded to 1. Removed experts are counted as zero bits.

There is a point here that is easy to misread: no single weight is stored at below 1 bit. The 280.1B parameters retained by Coder average 2.06 bits—it's just that the pruned-away parameters are counted as zero, which pulls down the overall average. This accounting convention is not uncommon in the compression community, but readers should indeed take note.

In addition, the shared Engram file (110.6 GB) and the vision projector are identical to the unpruned version, so they only need to be downloaded once.

GSQ and RCO: Breaking Down the Two Compression Techniques

GSQ (Gumbel-Softmax Quantization) is a post-training scalar quantization method that jointly learns grid assignments and group scales, narrowing the gap between scalar quantization and vector quantization at low bit widths while remaining deployable as a standard GGUF type.

RCO (Riemannian Constrained Optimization) precisely satisfies budget constraints through gradient descent on the task loss, without the need to tune a separate parameter for each constraint. In this project, RCO is responsible for selecting which experts to retain, with the optimization objective being the KL divergence relative to the unpruned model and the constraint being an exact budget per layer.

Why Can Each Layer Only Retain the Same Number of Experts?

The GGUF format stores a single expert count for the entire model, so each layer must retain exactly the same number of experts; imbalance across layers would lead to padding, which would offset the memory savings from pruning. RCO treats the per-layer budget as a hard constraint, maintaining the budget exactly at every optimization step across all layers, while all layers are jointly optimized through a single loss over the full model. In contrast, penalty-based methods can only approximately satisfy the budget and require tuning a coefficient for each layer.

The difference between scalar quantization and vector quantization: Quantization is essentially representing weights with fewer bits. Scalar quantization independently rounds each weight to some codebook, which is simple to implement and deployable with standard GGUF types; vector quantization maps a group of weights as a whole to a vector in the codebook, which can exploit correlations between weights and typically yields smaller error at the same bit count, but codebook lookup and deployment formats are more complex. The value of GSQ lies in narrowing the gap between the two while retaining the deployment convenience of scalar quantization.

The role of Gumbel-Softmax: Grid assignment is essentially a discrete choice (which quantization grid point each weight belongs to), and discrete choices are non-differentiable, so they cannot be directly backpropagated. Gumbel-Softmax is a reparameterization trick that approximates discrete sampling with a continuous relaxation parameterized by temperature, allowing gradients to flow back through the selection operation, so that "choosing grid points" and "learning group scales" can be jointly performed in the same post-training optimization process.

The storage structure of GGUF: GGUF is the model file format used by llama.cpp. It organizes weights in units of tensors and records the model's global structural hyperparameters in metadata, among which the expert count is a single model-level field rather than being variable per layer. Therefore, if the number of experts retained differs across layers, the loader can only pad according to the largest layer, and the padded empty slots also occupy memory, partially offsetting the space saved by pruning.

Structured pruning and budget constraints: Pruning entire groups of experts is structured pruning, which is easier to translate into actual speedups on general-purpose inference engines than unstructured sparsity. RCO's "hard constraint" means projecting parameters back into the feasible region after each gradient update, so that the budget always holds exactly; penalty-based methods instead write the constraint into the loss, can only approximately satisfy it at the end of optimization, and require different penalty coefficients for different layers, resulting in high tuning costs.

Performance: Code Generation Nearly Lossless, Agentic Tasks See Some Decline

Two benchmark results were officially announced:

• LiveCodeBench v6: 88.57, compared to 89.71 for the base model, retaining 98.7%
• SWE-bench Verified: 77.00, compared to 81.20 for the base model, retaining 94.8%

SWE-bench Verified requires running long-horizon agentic trajectories on real code repositories, and has lower tolerance for capacity reduction than single-problem code generation—which is consistent with the expected behavior when errors accumulate over multiple rounds. In other words, pruning has little impact on "writing a snippet of code," but a more noticeable impact on "performing multi-step operations on a real repository."

The official release also showcases three HTML examples generated from this Coder single file: a pelican riding a bicycle animation SVG, a three.js voxel pagoda garden, and a playable first-person voxel sandbox.

The difference between the two benchmarks: LiveCodeBench is a competitive-style algorithm problem benchmark with clear inputs and outputs and single-round solving; the model only needs to write a correct program in one generation, with no opportunity for error accumulation. SWE-bench Verified, on the other hand, draws issues from real open-source repositories and requires the model to locate files, modify code, and pass tests across multiple rounds of tool calls; small deviations at a single step are amplified along the trajectory, making it more sensitive to model capacity. This also explains why, despite both being code tasks, the retention rates differ by nearly four percentage points.

KL divergence as the pruning objective: RCO uses KL divergence relative to the unpruned model as the optimization objective, measuring the difference in output distribution between the retained model and the original model, rather than directly optimizing downstream task accuracy. The advantage of this approach is that it can be jointly optimized over the full model without task annotations; the cost is that a good distribution fit does not guarantee that every specific capability is equally retained.

How to Use and Where to Get It

The GGUF model can run in llama.cpp b11870 or newer, as well as in Strata. The repository includes the complete per-tensor RCO allocation scheme.

• Coder (expert-pruned version): ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF on Hugging Face
• Quantized model (unpruned): ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF
• GSQ paper: arxiv.org/abs/2604.18556, code: github.com/IST-DASLab/GSQ
• RCO paper: arxiv.org/abs/2605.00649, code: github.com/IST-DASLab/RCO

The official team positions the Coder build as an experimental release and welcomes feedback, especially regarding capabilities not covered by the calibration mixture data. They also accept requests to quantize or prune other models.

What This Release Means

Compressing a 763B-class MoE model down to 72 GB of transformer weights means that consumer-grade or single-machine multi-GPU environments now have the possibility of running very large models—although the 110.6 GB Engram file remains a non-negligible additional overhead. The combination of GSQ and RCO provides a reproducible technical path: the former addresses the accuracy loss of low-bit quantization, while the latter solves the budget allocation problem of structured pruning through hard constraints. For developers focused on open-source inference deployment and model compression, this is an experimental direction worth tracking.

The role of Engram: The shared Engram file mentioned in the article is a part of the model structure consisting of additional parameters, stored separately from the transformer weights. It did not participate in this round of pruning and quantization, so its size remains unchanged at 110.6 GB. This means that "72 GB of transformer weights" does not equal the total footprint of a fully runnable model; in actual deployment, VRAM must still be reserved for Engram or strategies such as CPU offloading must be adopted. When assessing hardware requirements, both parts should be calculated together.

One-line Summary

ISTA used GSQ plus RCO to halve and quantize a 763B MoE model, averaging 1 bit per parameter.

Paragraph Summary

ISTA Deep Algorithms and Systems Lab released a Coder build of DeepSeek-V4.1-Flash-1017, using two techniques—GSQ post-training scalar quantization and RCO hard-constraint optimization—to prune the 384 routed experts per layer down to 192, reducing transformer weights from 138.2 GB to 72.1 GB. The officially stated "1 bit per parameter" is an overall average: pruned experts are counted as zero bits, and the 280.1B parameters actually retained average 2.06 bits. In terms of performance, LiveCodeBench v6 retains 98.7% of the base model and SWE-bench Verified retains 94.8%, showing that long-horizon agentic tasks are more sensitive to capacity reduction. The model can run in llama.cpp b11870+ and Strata, but still requires downloading an additional 110.6 GB shared Engram file.