SetTensorScaleA

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product / Atlas A3 inference product	x
Atlas A2 training product / Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

In the MxMatmul scenario, sets the quantization coefficient matrix scaleA of the left matrix in matrix multiplication.

Prototype

      
           __aicore__ inline void SetTensorScaleA(const GlobalTensor<ScaleT>& gm, bool isTransposeScaleA = false);

      
           __aicore__ inline void SetTensorScaleA(const LocalTensor<ScaleT>& leftMatrix, bool isTransposeScaleA = false);

Parameters

**Table 1** Parameters
Parameter	Input/Output	Description
gm	Input	Matrix scaleA of the quantization coefficient. The type is GlobalTensor. For the Atlas 350 Accelerator Card, the supported data type is fp8_e8m0_t.
leftMatrix	Input	Matrix scaleA of the quantization coefficient. The type is LocalTensor, and TPosition can be TSCM or VECOUT. For the Atlas 350 Accelerator Card, the supported data type is fp8_e8m0_t.
isTransposeScaleA	Input	Whether matrix scaleA is transposed. The options are as follows: false (default): Matrix scaleA is not transposed. true: Matrix scaleA is transposed. Notes: If matrix scaleA is in NZ format, this parameter can only be set to false. If SCALE_ISTRANS of matrix scaleA is set to true, this parameter can be set to either true or false at runtime, allowing for alternating use of transposed and non-transposed matrix scaleA, except when matrix scaleA is in NZ format. If SCALE_ISTRANS of matrix scaleA is set to false, this parameter can only be set to false. If it is forcibly set to true, the precision will be abnormal. With the bias input, to ensure that the L1 Buffer space calculation size is consistent between the tiling side and the kernel side, and to guarantee correct result precision, the value of this parameter must be the same as the value of SCALE_ISTRANS defined for matrix A's MatmulTypeWithScale on the kernel side and the value of isScaleTrans of the SetScaleAType() API on the tiling side. In other words, all three parameters must be set to true or false simultaneously.

Returns

None

Restrictions

The size of the input scaleA address space must be greater than or equal to singleCoreM x singleCoreK/32.
This API is not supported when enableMixDualMaster (dual-master mode) is set to true.

Examples

      
           REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm, &tiling);
mm.SetTensorA(gm_a);
mm.SetTensorB(gm_b);
mm.SetTensorScaleA(gm_scaleA);    // Set the quantization coefficient matrix scaleA of the left matrix.
mm.SetTensorScaleB(gm_scaleB);
if (tiling.isBias) {
    mm.SetBias(gmBias);
}
mm.IterateAll(gm_c);
mm.End();

Parent topic: Matmul Kernel APIs