q=clamp⁡(⌊x/s⌉+z)q = \operatorname{clamp}(\lfloor x/s \rceil + z)

Integer quantization

Converts a real number x into a small integer q: divide by a scale s, round to the nearest integer, add a zero-point z, and clip to the allowed range.

The scale ss is how much real value one integer step stands for. The zero-point zz is the integer that stands for exactly 0.0. It must be exact, or every zero in the tensor (including padding) would come back slightly off.

The mixed brackets ⌊⋅⌉\lfloor \cdot \rceil mean round to the nearest integer. clamp⁡\operatorname{clamp} clips the result into the range the integer type can hold, from qmin⁡q_{\min} to qmax⁡q_{\max} (for example −128-128 to 127127 for 8-bit). To convert back, x^=s(q−z)\hat{x} = s(q - z). In general x^≠x\hat{x} \ne x, and keeping that error small is the whole design problem.

Example

x=0.37,  s=0.1,  z=0:q=⌊3.7⌉=4,x^=0.1×4=0.4x = 0.37,\; s = 0.1,\; z = 0: \quad q = \lfloor 3.7 \rceil = 4, \quad \hat{x} = 0.1 \times 4 = 0.4

The round trip loses 0.03.

Where the course uses it

Related terms