This section describes the communication operators supported by the
- Single-operator zero-copy: To reduce the memory copy overhead, HCCL can directly operate the memory transferred by the service to improve communication performance.
- Communication operator retry: When the communication is intermittently interrupted due to a network fault, HCCL attempts to retry the communication operator to improve communication stability.
- Deterministic computing: The same output is generated if a reduction communication operator is executed for multiple times with the same hardware and input.
- The following tables list the communication operators supported in expansion mode. Modes not listed are unsupported.
- In tables of this section, "√" indicates supported, "x" indicates unsupported, and "N/A" indicates not involved.
- Operators and network modes not listed are not supported.
AI CPU
Operator |
Network Mode |
Single-Operator Zero-Copy |
Deterministic Computing |
Retry |
Intra-Rank Communication |
Intra-SuperPoD Communication |
Inter-SuperPoD Communication |
|---|---|---|---|---|---|---|---|
Broadcast |
Single-operator |
√ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
AllGather |
Single-operator |
√ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
AllGatherV |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Reduce |
Single-operator |
☓ |
√ |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
AllReduce |
Single-operator |
√ |
√ |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
Scatter |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
ReduceScatter |
Single-operator |
√ |
√ |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
ReduceScatterV |
Single-operator |
☓ |
√ |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
√ |
√ |
√ |
√ |
√ |
|
AlltoAll |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
AlltoAllV |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
AlltoAllVC |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Send |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Recv |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph (Ascend IR) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
|
BatchSendRecv |
Single-operator |
☓ |
N/A |
√ |
√ |
√ |
√ |
Graph capture (ACLGraph) |
N/A |
N/A |
√ |
√ |
√ |
√ |
AIV
Operator |
Network Mode |
Single-Operator Zero-Copy |
Deterministic Computing |
Retry |
Intra-Rank Communication |
Intra-SuperPoD Communication |
Inter-SuperPoD Communication |
|---|---|---|---|---|---|---|---|
Broadcast |
Single-operator |
☓ |
N/A |
☓ |
√ |
☓ |
☓ |
Graph (Ascend IR) |
☓ |
N/A |
☓ |
☓ |
☓ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
N/A |
☓ |
☓ |
☓ |
☓ |
|
AllGather |
Single-operator |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
AllReduce |
Single-operator |
☓ |
√ |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
√ |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
√ |
☓ |
√ |
√ |
☓ |
|
ReduceScatter |
Single-operator |
☓ |
√ |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
√ |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
√ |
☓ |
√ |
√ |
☓ |
|
AlltoAll |
Single-operator |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
AlltoAllV |
Single-operator |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
AlltoAllVC |
Single-operator |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
AllGather |
Single-operator |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
ReduceScatterV |
Single-operator |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
Graph (Ascend IR) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
|
Graph capture (ACLGraph) |
☓ |
N/A |
☓ |
√ |
√ |
☓ |
The AIV mode features better performance in communication with a small amount of data and is mainly used in inference scenarios. In this mode:
- The single-operator zero-copy feature introduces memory negotiation during execution, which increases the communication latency. Therefore, it is not supported in AIV mode.
- The operator retry feature increases the execution time. Therefore, it is not supported in AIV mode.
- Only intra-SuperPoD communication is supported, while inter-SuperPoD communication is not supported.