SetAntiQuantVector
Function Description
Matmul computation supports half inputs of matrix A and int8 inputs of matrix B. In this scenario, the pseudo-quantization API must be called for pseudo-quantization. After the API is called, the pseudo-quantization operation is performed to convert matrix B to the half type when data is moved from the GM to L1. The pseudo-quantization API in this section provides a quantization parameter vector. The shape of the vector is [1, N]. The value of N is the same as that in M/N/K during Matmul matrix computation. Each row of matrix B is pseudo-quantized by using the pseudo-quantization coefficient in the corresponding row in the provided vector.
Call this API before calling Iterate or IterateAll.
Prototype
1 | __aicore__ inline void SetAntiQuantVector(const LocalTensor<SrcT> &offsetTensor, const LocalTensor<SrcT> &scaleTensor) |
Parameters
Parameter |
Input/Output |
Description |
|---|---|---|
offsetTensor |
Input |
Parameter vector for pseudo-quantization, which is used for addition |
scaleTensor |
Input |
Parameter vector for pseudo-quantization, which is used for multiplication |
Returns
None
Availability
Precautions
None