List of Supported Algorithms for Inter-Supernode Communication

The inter-supernode communication algorithm support tables described in this section apply only to the Atlas A3 training product/Atlas A3 inference product.

Ring algorithm

Collective Communication Operator	Data Type	Network Running Mode	Deterministic Computation	Handling Method for Unsupported Operators
ReduceScatter	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR or H-D_R algorithm is automatically selected.
AllGather	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR or H-D_R algorithm is automatically selected.
AllReduce	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR or H-D_R algorithm is automatically selected.
Reduce	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR or H-D_R algorithm is automatically selected.
ReduceScatterV	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The NHR or H-D_R algorithm is automatically selected.
Scatter	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The NHR or H-D_R algorithm is automatically selected.
AllGatherV	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The NHR or H-D_R algorithm is automatically selected.

H-D_R algorithm

Collective Communication Operator	Data Type	Network Running Mode	Deterministic Computation	Handling Method for Unsupported Operators
AllReduce	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR or ring algorithm is automatically selected.
Broadcast	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR or ring algorithm is automatically selected.
Reduce	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR or ring algorithm is automatically selected.

NHR algorithm

Collective Communication Operator	Data Type	Network Running Mode	Deterministic Computation	Handling Method for Unsupported Operators
ReduceScatter	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The H-D_R or ring algorithm is automatically selected.
AllGather	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The H-D_R or ring algorithm is automatically selected.
AllReduce	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The H-D_R or ring algorithm is automatically selected.
Broadcast	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The H-D_R or ring algorithm is automatically selected.
ReduceScatterV	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The H-D_R or ring algorithm is automatically selected.
Scatter	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The H-D_R or ring algorithm is automatically selected.
AllGatherV	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The H-D_R or ring algorithm is automatically selected.

NB algorithm

Collective Communication Operator	Data Type	Network Running Mode	Deterministic Computation	Handling Method for Unsupported Operators
ReduceScatter	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.
AllGather	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.
AllReduce	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.
Broadcast	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode Graph mode (Ascend IR)	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.
ReduceScatterV	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.
Scatter	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.
AllGatherV	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.

Pipeline algorithm

Collective Communication Operator	Data Type	Network Running Mode	Deterministic Computation	Handling Method for Unsupported Operators
AllGather	int8, int16, int32, int64, float16, float32, and bfp16	Single-operator mode (valid only when zero-copy is enabled)	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.
ReduceScatter	int8, int16, int32, float16, float32, and bfp16	Single-operator mode (valid only when zero-copy is enabled)	Supported	The NHR, H-D_R, or ring algorithm is automatically selected.

Parent topic: References