Independent running mechanism of AIC and AIV

Overview

The independent running mechanism of AIC and AIV is also called the dual-master mode. In separation mode, the AIC runs independently of the AIV, and is not driven by messages. This is different from the MIX mode (including matrix and vector computation), which drives the AIC to run by using the message mechanism. Enabling the dual-master mode can improve the Matmul computation performance. By default, the dual-master mode is disabled. You can enable it by setting the enableMixDualMaster parameter in MatmulConfig.

Use Case

When the matrix computation and vector computation code in the operator runs independently and is not driven by messages, you can enable the dual-master mode to improve the Matmul computation performance.

Restrictions

• This function supports only the Norm template and MDL template.
• The type of the operator kernel function is MIX, and the ratio of the number of AIC cores to the number of AIV cores is 1:1.
• The type of the operator kernel function is MIX, the ratio of the number of AIC cores to the number of AIV cores is 1:2, and the IBSHARE parameter is enabled for both matrices A and B.

• The values of this parameter for all Matmul objects in the same operator must be the same.
• Matrices A, B, and Bias can only be input from the global memory.
• The matrix computation result can be output to GlobalTensor only by calling the IterateAll API. That is, the computation result is stored in the address of the global memory, and cannot be obtained by calling the GetTensorC API.

Examples

For a complete operator example, see operator sample for enabling the active-active mode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// Set the template parameter enableMixDualMaster to true to enable the dual-master mode for the Norm template. For the MDL template, use the GetMDLConfig API to obtain the template parameters.
constexpr static MatmulConfig MM_CFG = GetNormalConfig(false, false, false, BatchMode::BATCH_LESS_THAN_L1, true, IterateOrder::ORDER_M, ScheduleType::OUTER_PRODUCT, false, true/*enableMixDualMaster*/);
Matmul<A_TYPE, B_TYPE, C_TYPE, BIAS_TYPE, MM_CFG> mm;

// Common Matmul computation.
REGIST_MATMUL_OBJ(&pipe, GetSysWorkSpacePtr(), mm);
mm.SetTensorA(gm_a);
mm.SetTensorB(gm_b);
mm.SetBias(gm_bias);
mm.IterateAll(gm_c);