GetMatmulApiTiling

Applicability

Function

Obtains the constant Matmul Tiling parameters at compile time.

The Matmul Tiling constantization feature allows retrieving constant tiling parameters during compilation and compiling the operator with them, thereby reducing scalar computation overhead and improving overall operator performance. Specifically, when obtaining a Matmul template, you can determine the single-core shape parameters (singleCoreM, singleCoreN, singleCoreK) and the base shape parameters (basicM, basicN, basicK) in MatmulConfig, or only determine the base shape parameters. By specifying single-core shape and base shape parameters, or only base shape parameters in the template acquisition API, you can obtain a custom template. Then, by calling this interface, you get the constantized Matmul Tiling parameters.

When the API to obtain a MatmulConfig template is called, if you only set baseM, baseN, and baseK to constants (referred to as partial constants), singleCoreM, singleCoreN, and singleCoreK remain at their default value (0), tiling is still required for initializing the Matmul object on the Kernel with REGIST_MATMUL_OBJ. If baseM, baseN, baseK, singleCoreM, singleCoreN, and singleCoreK are set to constants (referred to as full constants), a null pointer can be used in the place where the tiling parameters are passed to REGIST_MATMUL_OBJ.

After partial or full constants, the MatmulConfig template with constant parameters is obtained. Then, this API is used for constant tiling parameters. The return values of this API include the constant Matmul tiling parameters and MatmulConfig template.

Prototype

1
2

template<class A_TYPE, class B_TYPE, class C_TYPE, class BIAS_TYPE>
__aicore__ constexpr MatmulApiStaticTiling GetMatmulApiTiling(const MatmulConfig& mmCFG, int32_t l1Size = Impl::L1_SIZE)

Parameters

Returns

MatmulApiStaticTiling

Restrictions

• When the API is called to obtain a MatmulConfig template, set (baseM, baseN, and baseK) or (baseM, baseN, baseK, singleCoreM, singleCoreN, and singleCoreK) to constants, which must be consistent with the calculated values of Tiling.
• In the Batch Matmul scenario, full constants are supported, but a null pointer cannot be used to replace the input parameter Tiling of REGIST_MATMUL_OBJ.

Example

For details about the complete operator sample, see operator sample for Matmul static tiling.

1
2
3
4
5
6
7
8

// Define the Matmul object.
typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, half> aType; 
typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, half> bType; 
typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> cType; 
typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> biasType; 
// Use the GetNormalConfig API to obtain CFG and set the known singleshape information and baseM, baseN, and baseK to values the same as those of Tiling during runtime.
constexpr auto staticTiling = GetMatmulApiTiling<aType, bType, cType, biasType>(CFG, 524288); // The available size of the L1 buffer in this example is 512 KB.
AscendC::Matmul<aType, bType, cType, biasType, staticTiling > mm;