Pruning

Quantization is a model compression technique that reduces model parameters from high-precision floating-point numbers to integer representations of lower bit-width

模型压缩涵盖量化、剪枝、知识蒸馏三条主要技术路径

Model acceleration aims to reduce the number of sampling steps, and typical methods include DDIM and distillation methods.

Combining quantization and pruning can shrink model size by one to two orders of magnitude, at the cost of reduced generation quality.

Pruning removes weights or entire channels that contribute little to the output, producing sparse structures to reduce computation.

Lightweighting focuses on reducing parameter count and computation, and typical methods include pruning and quantization.

Large model lightweighting techniques include model distillation, quantization (such as INT8/INT4), pruning, and on-device inference framework optimization, among other methods.

The practical significance of the average conclusion that quantization outperforms pruning depends on the type of capability the application requires; if the application relies on capabilities that pruning damages more severely, the average conclusion has no guiding value.

Pruning directly removes some weights or neurons, changing the sparse structure of the network.

Quantization errors are usually distributed uniformly across all parameters, whereas the destructiveness of pruning is more concentrated on the removed connections, so the two have different patterns of impact on knowledge distribution.

Quantization systematically outperforms pruning in capability retention.

Pruning, quantization, and knowledge distillation belong to model-side compression, which changes the parameter representation of the deployed model, while input vectors still need to be transmitted and stored at original precision.

Migrating a model from cloud inference to local operation typically requires compression steps such as quantization, pruning and distillation.

Quantization, pruning, and low-rank decomposition are three common means of model compression.

Distillation, quantization, and pruning are often collectively called the three-piece model compression toolkit, but distillation changes the training signal, while the latter two directly compress existing weights.

Pruning compresses model size by removing neurons or attention heads that contribute little to the output.

Pruning streamlines the network structure by removing weights or neurons that contribute less to the output.

Over the past two years, techniques such as model quantization, pruning, and knowledge distillation have allowed hundred-billion-parameter models to run on a single consumer-grade graphics card or even on a phone.

剪枝和量化在保持原架构拓扑的前提下做减法，不改变层的功能语义和连接模式；而Priming允许目标架构在层数、宽度、注意力模式上与原模型不同。

Pruning is the process of identifying and removing neural network weights or entire channels that contribute minimally to the output, thereby reducing the model's structure.