Selecting Algorithms

Selecting a Policy

When the communication engine is AIV, communication operators are expanded on Vector Cores. This results in shorter paths and lower latency overhead during the dispatch and expansion, making it ideal for latency-sensitive inference scenarios with small communication data volumes.

While a communication operator typically supports multiple algorithmic implementations, the mesh algorithm is highly recommended for AIV communication operators. This is because the Mesh algorithm requires minimal communication steps, which can further reduce communication latency.

Figure 1 AIV engine communication algorithm selection

1. If the communication operator has only one communication engine and one algorithm implementation, skip the algorithm selection step in this section.
2. The algorithms described in this section are implemented by yourself. The algorithms configured by the HCCL_ALGO environment variable are built-in HCCL algorithms. For details about the built-in HCCL algorithms, see in Huawei Collective Communication Library (HCCL)Introduction to Collective Communication Algorithms.

Sample Code

Take the algorithm selection policy in Selecting a Policy as an example. The corresponding algorithm selection code snippet is as follows:

1
2
3
4
5
6

CommEngine engine;
if (engine == CommEngine::COMM_ENGINE_AIV) {
    algName = "AivMesh"; // Use the mesh algorithm expanded by AIV.
} else {
    return HCCL_E_NOT_SUPPORT;
}