SetTensorScaleB

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product / Atlas A3 inference product	x
Atlas A2 training product / Atlas A2 inference product	x
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

In the MxMatmul scenario, sets the quantization coefficient matrix scaleB of the right matrix in matrix multiplication.

Prototype

      
           __aicore__ inline void SetTensorScaleB(const GlobalTensor<ScaleT>& gm, bool isTransposeScaleB = true);

      
           __aicore__ inline void SetTensorScaleB(const LocalTensor<ScaleT>& rightMatrix, bool isTransposeScaleB = true);

Parameters

**Table 1** Parameters
Parameter	Input/Output	Description
gm	Input	Matrix scaleB of the quantization coefficient. The type is GlobalTensor. For the Atlas 350 Accelerator Card, the supported data type is fp8_e8m0_t.
rightMatrix	Input	Matrix scaleB of the quantization coefficient. The type is LocalTensor, and TPosition can be TSCM or VECOUT. For the Atlas 350 Accelerator Card, the supported data type is fp8_e8m0_t.
isTransposeScaleB	Input	Whether matrix scaleB is transposed. The options are as follows: false: Matrix scaleB is not transposed. true (default): Matrix scaleB is transposed. Notes: If matrix scaleB is in NZ format, this parameter can only be set to true. If SCALE_ISTRANS of matrix scaleB is set to true, this parameter can be set to either true or false at runtime, allowing for alternating use of transposed and non-transposed matrix scaleB, except when matrix scaleB is in NZ format. If SCALE_ISTRANS of matrix scaleB is set to false, this parameter can only be set to false. If it is forcibly set to true, the precision will be abnormal. With the bias input, to ensure that the L1 Buffer space calculation size is consistent between the tiling side and the kernel side, and to guarantee correct result precision, the value of this parameter must be the same as the value of SCALE_ISTRANS defined for matrix B's MatmulMxType on the kernel side and the value of isScaleTrans of the SetScaleBType() API on the tiling side. In other words, all three parameters must be set to true or false simultaneously.

Returns

None

Restrictions

The size of the input scaleB address space must be greater than or equal to singleCoreK x singleCoreN/32.
This API is not supported when enableMixDualMaster (dual-master mode) is set to true.

Examples

      
           REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm, &tiling);
mm.SetTensorA(gm_a);
mm.SetTensorB(gm_b);
mm.SetTensorScaleA(gm_scaleA);
mm.SetTensorScaleB(gm_scaleB);    // Set the quantization coefficient matrix scaleB of the right matrix.
if (tiling.isBias) {
    mm.SetBias(gmBias);
}
mm.IterateAll(gm_c);
mm.End();

Parent topic: Matmul Kernel APIs