Matmul Template Parameters

Applicability

Function

The following parameters need to be passed to create a Matmul object:

• Parameter types of matrices A, B, and C, and the bias. The type information is defined by MatmulType, including the logical memory location, data format, data type, whether to transpose, data layout, and whether to enable L1 reuse.
• (Optional) MatmulConfig information, which is used to configure Matmul template information and related parameters. If it is not set, the Norm template is enabled by default.

  For the Atlas 200I/500 A2 inference products, only the default Norm template can be enabled.

• (Optional) MatmulCallBackFunc custom function information, which is used to configure a function for copying the left and right matrices from GM to L1 and copying the computation result from CO1 to GM. The following product models are supported:

  Atlas A3 training products/Atlas A3 inference products

  Atlas A2 training products/Atlas A2 inference products

• (Optional) MatmulPolicy information, which is used to configure the Matmul extensible module policy. If this parameter is not set, the default template policy is enabled. The following product models are supported:

  Atlas A3 training products/Atlas A3 inference products

  Atlas A2 training products/Atlas A2 inference products

  Atlas 200I/500 A2 inference products

  Atlas inference product's AI Core

Prototype

1
2

template <class A_TYPE, class B_TYPE, class C_TYPE, class BIAS_TYPE = C_TYPE, const auto& MM_CFG = CFG_NORM, class MM_CB = MatmulCallBackFunc<nullptr, nullptr, nullptr>, MATMUL_POLICY_DEFAULT_OF(MatmulPolicy)>
using Matmul = AscendC::MatmulImpl<A_TYPE, B_TYPE, C_TYPE, BIAS_TYPE, MM_CFG, MM_CB, MATMUL_POLICY>;

• The A_TYPE, B_TYPE, and C_TYPE types are defined by MatmulType.
• (Optional) MM_CFG parameter of the auto type:
  • The MatmulConfig type is supported:

    For details about the Matmul template information, see MatmulConfig.

  • The MatmulApiStaticTiling type is supported:

    For details about MatmulApiStaticTiling parameters, see Table 1.

    The MatmulApiStaticTiling structure contains a group of constant Tiling parameters and a MatmulConfig structure. To define this type of parameters, call the API for obtaining the template described in MatmulConfig and specify the (singleM, singleN, singleK, baseM, baseN, baseK) parameters to obtain a custom template. Then, pass the template to the GetMatmulApiTiling API to obtain the constant parameters. This method produces a group of constant parameters defined in the MatmulApiStaticTiling structure for optimizing scalar operations inside the Matmul computation. Currently, the following templates that support constant Tiling parameters via MatmulApiStaticTiling: Norm, IBShare, and MDL.

• (Optional) MM_CB is used to support different transfer-in and transfer-out requirements and implement customized transfer-in and transfer-out functions. For details, see MatmulCallBackFunc.
• (Optional) MATMUL_POLICY_DEFAULT_OF(MatmulPolicy), which is used to configure the Matmul extensible module policy. Currently, this parameter can be left empty (the default template policy is enabled) or one MatmulPolicy parameter can be configured.
  The definition of MATMUL_POLICY_DEFAULT_OF is as follows, which is used to simplify the type declaration of MATMUL_POLICY. For details about how to use the template parameters, see MatmulPolicy.

  1 2 3

  #define MATMUL_POLICY_DEFAULT_OF(DEFAULT) \ template <const auto& = MM_CFG, typename ...> \ class MATMUL_POLICY = AscendC::Impl::Detail::DEFAULT

Parameters

Returns

None

Restrictions

None

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

// User-defined callback function
void DataCopyOut(const __gm__ void *gm, const LocalTensor<int8_t> &co1Local, const void *dataCopyOutParams, const uint64_t tilingPtr, const uint64_t dataPtr);
void CopyA1(const AscendC::LocalTensor<int8_t> &aMatrix, const __gm__ void *gm, int row, int col, int useM, int useK, const uint64_t tilingPtr, const uint64_t dataPtr);
void CopyB1(const AscendC::LocalTensor<int8_t> &bMatrix, const __gm__ void *gm, int row, int col, int useK, int useN, const uint64_t tilingPtr, const uint64_t dataPtr);

typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, half> aType; 
typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, half> bType; 
typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> cType; 
typedef AscendC::MatmulType<AscendC::TPosition::GM, CubeFormat::ND, float> biasType; 
AscendC::Matmul<aType, bType, cType, biasType, CFG_MDL> mm1; 
AscendC::MatmulConfig mmConfig{false, true, false, 128, 128, 64};
mmConfig.enUnitFlag = false;
AscendC::Matmul<aType, bType, cType, biasType, mmConfig> mm2;
AscendC::Matmul<aType, bType, cType, biasType, CFG_NORM, AscendC::MatmulCallBackFunc<DataCopyOut, CopyA1, CopyB1>> mm3;