Post-training quantization algorithms such as GPTQ and AWQ can compress models to 4-bit INT4, enabling 70B-parameter models to run on consumer-grade GPUs with a reduction in computation of more than 75%.