Independent Running Mechanism of AIC and AIV

Overview

The independent running mechanism of AI Cube (AIC) and AI Vector (AIV) is also called the dual-master mode. In the separated mode, the dual-master mode enables the AIC and AIV to run independently without depending on the message mechanism, thus enhancing the Matmul computation performance. This is different from the MIX mode (including cube computation and vector computation) that drives the AIC to run using the message mechanism. By default, the dual-master mode is disabled. You can enable it by setting the enableMixDualMaster parameter in MatmulConfig.

Application Scenarios

When the cube computation and vector computation code in an operator runs independently and is not driven by the message mechanism, you can enable the dual-master mode to enhance the Matmul computation performance.

Constraints

• This function applies to the Norm template and MDL template only.
• The type of the operator kernel function is MIX, and the ratio of AIC Cores to AIV Cores is 1:1.
• The type of the operator kernel function is MIX, the ratio of AIC Cores to AIV Cores is 1:2, and the IBSHARE parameter is enabled for both matrix A and matrix B.

• The value of the enableMixDualMaster parameter must be the same for all Matmul objects in the same operator.
• Matrix A, matrix B, and the bias matrix can be input from the global memory only.
• For the matrix computation result, only the IterateAll API can be called to output the result to GlobalTensor. In other words, the computation result is stored in the address of the global memory. No other APIs, such as GetTensorC, can be called.

Example

For a complete operator example, see operator sample for enabling the active-active mode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// Change the value of template parameter enableMixDualMaster to true to enable the dual-master mode for the Norm template. For the MDL template, use the GetMDLConfig API to obtain the template parameters.
constexpr static MatmulConfig MM_CFG = GetNormalConfig(false, false, false, BatchMode::BATCH_LESS_THAN_L1, true, IterateOrder::ORDER_M, ScheduleType::OUTER_PRODUCT, false, true/*enableMixDualMaster*/);
Matmul<A_TYPE, B_TYPE, C_TYPE, BIAS_TYPE, MM_CFG> mm;

// Regular Matmul computation.
REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm);
mm.SetTensorA(gm_a);
mm.SetTensorB(gm_b);
mm.SetBias(gm_bias);
mm.IterateAll(gm_c);