IterateAll

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Computes a matrix C of size singleCoreM x singleCoreN by each call to IterateAll. The iteration sequence can be adjusted using the tiling parameter iterateOrder.

Prototype

template <bool sync = true> __aicore__ inline void IterateAll(const GlobalTensor<DstT>& gm, uint8_t enAtomic = 0, bool enSequentialWrite = false, bool waitIterateAll = false, bool fakeMsg = false)

template <bool sync = true> __aicore__ inline void IterateAll(const LocalTensor<DstT>& ubCmatrix, uint8_t enAtomic = 0)

Parameters

**Table 1** Template parameters
Parameter	Description
sync	Matrix C can be obtained in synchronous or asynchronous mode. Synchronous mode: Wait until IterateAll is executed. Asynchronous mode: Do not need to wait until IterateAll is executed. Setting it to true (default) enables the synchronous mode; while setting it to false enables the asynchronous mode. For the Atlas 200I/500 A2 inference product, it can only be set to true.

**Table 2** API parameters
Parameter	Input/Output	Description
gm	Output	Matrix C. The type is GlobalTensor. For the Atlas 350 Accelerator Card, the supported data types are half, float, bfloat16_t, int32_t, int8_t, fp8_e4m3fn_t, and hifloat8_t. For the Atlas A3 training product/Atlas A3 inference product, the supported data types are half, float, bfloat16_t, int32_t, and int8_t. For the Atlas A2 training product/Atlas A2 inference product, the supported data types are half, bfloat16_t, float, int32_t, and int8_t. For the Atlas inference product AI Core, the supported data types are half, float, int8_t, and int32_t. For the Atlas 200I/500 A2 inference product, the supported data types are half, bfloat16_t, float, and int32_t.
ubCmatrix	Output	Matrix C. The type is LocalTensor, and TPosition can be TSCM or VECIN. You are advised to use the Iterate API for output to the VECIN tensor. For the Atlas 350 Accelerator Card, the supported data types are half, float, bfloat16_t, int32_t, int8_t, fp8_e4m3fn_t, and hifloat8_t. For the Atlas A3 training product/Atlas A3 inference product, the supported data types are half, float, bfloat16_t, int32_t, and int8_t. For the Atlas A2 training product/Atlas A2 inference product, the supported data types are half, bfloat16_t, float, int32_t, and int8_t. For the Atlas inference product AI Core, the prototype API that contains this parameter is not supported. For the Atlas 200I/500 A2 inference product, the supported data types are half, bfloat16_t, float, and int32_t.
enAtomic	Input	Enables the Atomic operation or not. Values: 0 (default): disables the Atomic operation. 1: enables the AtomicAdd (accumulation) operation. 2: enables the AtomicMax (maximum value calculation) operation. 3: enables the AtomicMin (minimum value calculation) operation. For the Atlas inference product AI Core, the Atomic operation can be enabled only when the output position is GM. For the Atlas 200I/500 A2 inference product, the Atomic operation can be enabled only when the output position is GM.
enSequentialWrite	Input	Enables the continuous write mode or not (write to [baseM,baseN] for continuous write and to [singleCoreM,singleCoreN] for discontinuous write). The default value is false (discontinuous write). This parameter is not supported by the Atlas 200I/500 A2 inference product.
waitIterateAll	Input	Used only in asynchronous scenarios, indicating whether to use WaitIterateAll to wait for the completion of IterateAll execution. true: WaitIterateAll is used to wait for the completion of IterateAll execution. false: WaitIterateAll is not used to wait for the completion of IterateAll execution. Developers can handle this waiting process themselves.
fakeMsg	Input	This parameter is used only in the IBShare scenario (doIBShareNorm is enabled in template parameters) and IntraBlockPartSum scenario (intraBlockPartSum is enabled in template parameters). IBShare scenario In this scenario, the same matrix A or matrix B data on L1 is reused. The number of times that AIV calls IterateAll by core must be matched. In this case, fakeMsg needs to be set to true, and no computation is performed. This ensures that IterateAll is called in pairs. The default value is false, indicating that computation is performed. IntraBlockPartSum scenario This scenario is used to enable the accumulation of computation results (matrix slices with the size of baseM × baseN) of multiple AIV cores on L0C Buffer in the case of fused vector and cube computation in separation mode. The default value is false, indicating that the Matmul computation results of each AIV core are accumulated on L0C Buffer.

Returns

None

Restrictions

Ensure that the size of the address space of the input matrix C is greater than or equal to singleCoreM × singleCoreN elements.

Example

The following is an example of calling the IterateAll API. For more operator examples in asynchronous scenarios, see IterateAll asynchronous scenario sample.

REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm, &tiling);
mm.SetTensorA(gm_a);
mm.SetTensorB(gm_b);
if (tiling.isBias) {
    mm.SetBias(gmBias);
}
mm.IterateAll(gm_c);    // Computation
mm.End();

Parent topic: Matmul Kernel APIs