IterateAll

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	√
Atlas inference product's AI Core	√
Atlas inference product's Vector Core	x
Atlas training products	x

Function

Computes a matrix C of size singleCoreM x singleCoreN by each call to IterateAll. The iteration sequence can be adjusted using the tiling parameter iterateOrder.

Prototype

template <bool sync = true> __aicore__ inline void IterateAll(const GlobalTensor<DstT>& gm, uint8_t enAtomic = 0, bool enSequentialWrite = false, bool waitIterateAll = false, bool fakeMsg = false)

template <bool sync = true> __aicore__ inline void IterateAll(const LocalTensor<DstT>& ubCmatrix, uint8_t enAtomic = 0)

Parameters

**Table 1** Parameters in the template
Parameter	Description
sync	Matrix C can be obtained in synchronous or asynchronous mode. Synchronous mode: Wait until IterateAll is executed. Asynchronous mode: Do not need to wait until IterateAll is executed. Setting it to true (default) enables the synchronous mode; while setting it to false enables the asynchronous mode. For the Atlas 200I/500 A2 inference products, it can only be set to true.

**Table 2** API parameters
Parameter	Input/Output	Description
gm	Output	Matrix C. The type is GlobalTensor. For Atlas A3 training products/Atlas A3 inference products, the supported data types are half, float, bfloat16_t, int32_t, and int8_t. For Atlas A2 training products/Atlas A2 inference products, the supported data types are half, bfloat16_t, float, int32_t, and int8_t. For the Atlas inference product's AI Core, the supported data types are half, float, int8_t, and int32_t. For Atlas 200I/500 A2 inference productss, the supported data types are half, bfloat16_t, float, and int32_t.
ubCmatrix	Output	Matrix C. The type is LocalTensor, and TPosition can be set to TSCM. For the Atlas A3 training products/Atlas A3 inference products, the supported data types are half, float, bfloat16_t, int32_t, and int8_t. For the Atlas A2 training products/Atlas A2 inference products, the supported data types are half, bfloat16_t, float, int32_t, and int8_t. For the Atlas inference product's AI Core, the prototype API that contains this parameter is not supported. For Atlas 200I/500 A2 inference productss, the supported data types are half, bfloat16_t, float, and int32_t.
enAtomic	Input	Enables the Atomic operation or not. Values: 0 (default): disables the Atomic operation. 1: enables the AtomicAdd (accumulation) operation. 2: enables the AtomicMax (maximum value calculation) operation. 3: enables the AtomicMin (minimum value calculation) operation. For the Atlas inference product's AI Core, the Atomic operation can be enabled only when the output position is GM. For the Atlas 200I/500 A2 inference products, the Atomic operation can be enabled only when the output position is GM.
enSequentialWrite	Input	Enables the continuous write mode or not (write to [baseM,baseN] for continuous write and to [singleCoreM,singleCoreN] for discontinuous write). The default value is false (discontinuous write). For Atlas 200I/500 A2 inference productss, this parameter is not supported.
waitIterateAll	Input	Used only in asynchronous scenarios, indicating whether to use WaitIterateAll to wait for the completion of IterateAll execution. true: WaitIterateAll is used to wait for the completion of IterateAll execution. false: WaitIterateAll is not used to wait for the completion of IterateAll execution. Developers can handle this waiting process themselves.
fakeMsg	Input	This parameter is used only in the IBShare scenario (doIBShareNorm is enabled in template parameters) and IntraBlockPartSum scenario (intraBlockPartSum is enabled in template parameters). IBShare scenario In this scenario, the same matrix A or matrix B data on L1 is reused. The number of times that AIV calls IterateAll by core must be matched. In this case, fakeMsg needs to be set to true, and no computation is performed. This ensures that IterateAll is called in pairs. The default value is false, indicating that computation is performed. IntraBlockPartSum scenario This scenario is used to enable the accumulation of computation results (matrix slices with the size of baseM × baseN) of multiple AIV cores on L0C Buffer in the case of fused vector and cube computation in separation mode. The default value is false, indicating that the Matmul computation results of each AIV core are accumulated on L0C Buffer.

Returns

None

Restrictions

Ensure that the size of the address space of the input matrix C is greater than or equal to the number of elements in singleCoreM × singleCoreN.

Example

The following is an example of calling the IterateAll API. For more operator examples in asynchronous scenarios, see matrix multiplication in IterateAll asynchronous scenarios.

REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm, &tiling);
mm.SetTensorA(gm_a);
mm.SetTensorB(gm_b);
mm.SetBias(gm_bias);
mm.IterateAll(gm_c);    // Computation

Parent topic: Matmul Kernel APIs