Documentation

GetNormalConfig

Function Usage

Configures the parameters to obtain the user-defined Norm template.

Prototype

1
__aicore__ constexpr MatmulConfig GetNormalConfig(const bool intrinsicsLimit = false, const bool batchLoop = false, const bool isVecND2NZ = false, const BatchMode bmmMode = BatchMode::BATCH_LESS_THAN_L1, const bool isMsgReuse = true, const IterateOrder iterateOrder = IterateOrder::UNDEF, const ScheduleType scheduleType = ScheduleType::INNER_PRODUCT, const bool enUnitFlag = true)

Parameters

All parameters of this API are used to set the parameters of the MatmulConfig structure. Parameters corresponding to each other have the same functions.

Table 1 API parameters

Parameter

Input/Output

Description

intrinsicsLimit

Input

Sets the intrinsicsCheck parameter.

Whether to enable cyclic data move-in when the inner axis (last axis) of the left or right matrix on a single core is greater than or equal to 65535. For example, for the left matrix A [M, K], if singleCoreK of the inner axis on a single core is greater than 65535 and this parameter is set to true, data is moved in cyclically in the API. Values:

  • false (default): When the inner axis of the left or right matrix on a single core is greater than or equal to 65535, data is not moved in cyclically.
  • true: When the inner axis of the left or right matrix on a single core is greater than or equal to 65535, data is moved in cyclically.

batchLoop

Input

Sets the isNBatch parameter.

Whether to enable multi-batch input and output. This parameter is valid only for BatchMatmul. Values:

  • false (default): disables multi-batch input and output.
  • true: enables the multi-batch function.

isVecND2NZ

Input

Sets the enVecND2NZ parameter.

Whether to enable ND2NZ (converting data from ND format to NZ format) using the vector. To enable this function, you need to set SetLocalWorkspace. Values:

  • false (default): disables ND2NZ using the vector.
  • true: enables ND2NZ using the vector.

bmmMode

Input

Sets the batchMode parameter.

Relationship between the total amount of multi-batch data for input matrices A and B in a BatchMatmul operation and the size of L1 Buffer when the layout mode is set to NORMAL. Values:

  • BatchMode::BATCH_LESS_THAN_L1: Total amount of multi-batch data < Size of L1 Buffer
  • BatchMode::BATCH_LARGE_THAN_L1: Total amount of multi-batch data > Size of L1 Buffer
  • BatchMode::SINGLE_LARGE_THAN_L1: Total amount of single-batch data > Size of L1 Buffer

isMsgReuse

Input

Sets the enableReuse parameter.

SetSelfDefineDatadirectly passes computation data. The parameter values are as follows:

  • true: passes computation data. Only a single value is supported.
  • false: passes data address information stored on GM.

iterateOrder

Input

Sets the iterateOrder parameter.

Iteration order for Matmul to perform matrix computation. The meaning of this parameter is the same as that of iterateOrder in Table 1. This parameter is valid only when ScheduleType is set to ScheduleType::OUTER_PRODUCT or 1. Values:

1
2
3
4
5
enum class IterateOrder {
    ORDER_M = 0,   // Offset to the M-axis direction and then to the N-axis direction.
    ORDER_N,       // Offset to the N-axis direction and then to the M-axis direction.
    UNDEF,         // Invalid currently.
};

Note: When the Norm template (Matmul scenario) and the MDL template are used, if IterateOrder is set to ORDER_M, the value of stepN in the TCubeTiling structure must be greater than 1; if IterateOrder is set to ORDER_N, the value of stepM in the TCubeTiling structure must be greater than 1.

scheduleType

Input

Sets the scheduleType parameter.

Matmul data movement mode. Values:

  • ScheduleType::INNER_PRODUCT or 0 (default): performs MTE1 cyclic movement in the K direction.
  • ScheduleType::OUTER_PRODUCT or 1: performs MTE1 cyclic movement in the M or N direction. This parameter must be used together with the IterateOrder parameter. Its configuration takes effect only in the Norm template (BatchMatmul and Matmul scenarios) and the MDL template.
    • If the value of IterateOrder is set to ORDER_M, cyclic movement is performed in the N direction, that is, data in matrix B is moved in parallel using MTE1. (The performance may be improved when the value of singleCoreN is greater than that of baseN.)
    • If the value of IterateOrder is set to ORDER_N, cyclic movement is performed in the M direction, that is, data in matrix A is moved in parallel using MTE1. (The performance may be improved when the value of singleCoreM is greater than that of baseM.)
    • The cyclic movement in the M direction and N direction cannot be enabled at the same time.

Note:

  • In the Norm template (BatchMatmul scenario) or the MDL template, when singleCoreK is greater than baseK, the value of ScheduleType::OUTER_PRODUCT cannot be enabled and the default mode must be used.
  • This parameter can be set to ScheduleType::OUTER_PRODUCT or 1 only when the MDL template calls IterateAll for computation.
  • This parameter can be set to ScheduleType::OUTER_PRODUCT or 1 only when matrix C is output to GM.

enUnitFlag

Input

Sets the enUnitFlag parameter.

Whether to enable the unitflag function to allow parallel execution of computation and data movement for performance improvement. By default, the function is enabled when the Norm and IBShare templates are used and disabled when the MDL template is used. Values:

  • false: disables the unitflag function.
  • true: enables the unitflag function.

Returns

MatmulConfig structure.

Availability

Precautions

None

Example

1
2
3
4
5
6
7
constexpr MatmulConfig MM_CFG = GetNormalConfig();
Matmul<A_TYPE, B_TYPE, C_TYPE, BIAS_TYPE, MM_CFG> mm;
REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm, &tiling);
mm.SetTensorA(gm_a);
mm.SetTensorB(gm_b);
mm.SetBias(gm_bias);
mm.IterateAll(gm_c);
Parent topic: Matmul