TDEQUANT

Introduction

Dequantizes a low-precision quantized tile (S8/S16) into a high-precision FP32 tile. Performs affine dequantization on each element, which is the inverse operation of the integer quantization (INT8 symmetric/asymmetric) performed by TQUANT:

$$\mathrm{dst}_{i,j} = (\mathrm{src}_{i,j} - \mathrm{offset}_{i}) \cdot \mathrm{scale}_{i}$$

scale and offset are FP32 parameters provided per row (broadcast along columns to the entire row), so the complete dequantization of "offset removal + inverse scaling" can be accomplished in a single instruction.

Mathematical Semantics

Conceptually, for each element (i, j) in the valid region:

$$\mathrm{dst}_{i,j} = \bigl(\mathrm{src}_{i,j} - \mathrm{offset}_{i}\bigr) \cdot \mathrm{scale}_{i}$$

• src: quantized integer code (S8 or S16).
• scale, offset: per-row FP32 dequantization parameters (the parameter group is selected by row index i and broadcast along columns to the entire row); the valid column count of scale is paraCols = max(1, scale.GetValidCol()), and the parameter column subscript paraCol = min(j, paraCols - 1) is used only to clamp the read column subscript when the parameter tile has multiple columns (typically one scalar per row, paraCols = 1); the parameter group itself is determined by row i.
• It is the inverse of the integer affine quantization performed by TQUANT: in TQUANT, $q = \mathrm{round}(x / \mathrm{scale}) + \mathrm{offset}$, hence $x = (q - \mathrm{offset}) \cdot \mathrm{scale}$.

Unless otherwise specified, the semantics are defined within the valid region, and target-dependent behavior is marked as implementation-defined. Both scale and offset are ISA-visible tile operands (not compiler scratch).

C++ Built-in APIs

Declared in include/pto/common/pto_instr.hpp.

The public include header is <pto/pto-inst.hpp>, and the internal declaration is located in pto/common/pto_instr.hpp.

template <typename TileDataDst, typename TileDataSrc, typename TileDataPara, typename... WaitEvents>
PTO_INST RecordEvent TDEQUANT(TileDataDst &dst, TileDataSrc &src, TileDataPara &scale, TileDataPara &offset,
                              WaitEvents &...events);

scale and offset must be of the same type (TileDataPara), and their dtypes must be consistent with dst (both FP32).

Tile Size and Data Types

For the valid shape $M \times N$ of the source/destination tiles:

The valid row count of scale/offset must equal the valid row count of dst; the column direction is broadcast in 32-byte blocks, so the typical usage is one scalar per row (shape $M \times 1$).

Supported Input Dtypes

dst, scale, and offset must have the same dtype, which must be FP32; src supports only S8/S16. Other dtype combinations are invalid (intercepted by static_assert in the implementation).

Implementation Notes

TDEQUANT executes on the vector pipe (PIPE_V) and does not require a tmp scratch tile (unlike the 5-stage type conversion chain of TQUANT on Atlas A2/A3 training products/Atlas A2/A3 inference products):

1. Load and unpack src: S8 is unpacked via UNPK4_B8, and S16 via UNPK_B16, then converted to FP32 via vcvt (on KirinX90, S8 takes the US_B8 + interleaved path).
2. Broadcast load parameters: scale and offset are broadcast to the entire row in 32-byte blocks via vlds ... BRC_B32.
3. Compute: vsub(dst, src, offset) followed by vmul(dst, dst, scale), that is, subtract the offset first and then de-scale.

Encoding

TDEQUANT belongs to the Tile Elementwise Pipeline (TEPL) compound transform class instruction:

BSTART.TEPL TDEQUANT, DataType +
B.DATR(optional) +
B.DIM LB0 +
B.DIM (LB1/LB2 for 2D) +
B.IOT

Constraints

Examples

// src: int8/int16 quantization code; scale/offset: FP32 parameters per row
TDEQUANT(dstTile, srcTile, scaleTile, offsetTile);

For complete ST examples, see tests/npu/a5/src/st/testcase/tdequant/ (A5), tests/npu/a2a3/src/st/testcase/tdequant/ (A2/A3), and tests/cpu/st/testcase/tdequant/ (CPU reference implementation).