Quantization

Short answer

Quantization is a technique for compressing an AI model by storing its weights at lower numerical precision, such as 8-bit or 4-bit numbers instead of 16-bit or 32-bit. This cuts memory use and computing cost, allowing models to run faster or on devices, usually with a modest trade-off in accuracy.