BroadcastOperation

Applicable Products

Hardware Model	Supported	Description
Atlas 350 accelerator cards	x	-
Atlas A3 inference products/Atlas A3 training products	√	-
Atlas A2 training products	√	-
Atlas A2 inference products	√	Only lccl is supported.
Atlas training products	√	Only hccl is supported.
Atlas inference products	x	-
Atlas 200I/500 A2 inference products	x	-

Description

Broadcast the data on the primary communication card to each of the other cards. This operator performs in-place operation. The memory address of the output tensor is the same as that of the input tensor.

This operator involves multi-device operations. You can configure HCCL environment variables as required. For details, see Collective Communication in CANN Environment Variables.

Application Scenarios

One-to-many broadcast. Primary node 0 sends the same data to other nodes.

Figure 1 Communication principle

Example

Parameter setting
```
rankRoot=0
rankSize=2
```

Input and output

npu0 (primary device):
x=[67,60,-75]
output=[67,60,-75]
npu1:
x=[0,0,0]
output=[67,60,-75]

Definition

struct BroadcastParam {
    int rank = 0;
    int rankSize = 0;
    int rankRoot = 0;
    HcclComm hcclComm = nullptr; 
    CommMode commMode = COMM_MULTI_PROCESS;
    std::string backend = "hccl";
    std::string rankTableFile;
    std::string commDomain;
    uint8_t rsv[64] = {0};
};

Parameters

Member	Type	Default Value	Description
rank	int	0	Rank of the current device within the communicator.
rankSize	int	0	Number of devices.
rankRoot	int	0	Root rank for the operation.
hcclComm	HcclComm	nullptr	Pointer to the HCCL communicator. By default, this parameter is left blank. The ATB is created by users. If the user wants to manage the communicator, the communicator pointer needs to be passed. The ATB uses the passed communicator pointer to execute the communication operator.
commMode	CommMode	COMM_MULTI_PROCESS	Communication mode. For details, see CommMode. In the scenario where hccl multi-thread is applied, a communicator can only be imported externally.
backend	string	hccl	Communication backend. hccl and lccl are supported. The LCCL function will be deleted in later versions. You are advised to use the HCCL function. When backend is set to hccl, Atlas A2 inference products are not supported. When backend is set to lccl: For Atlas 200T A2 Box16 heterogeneous subrack, the communicator can be created only among the first eight or last eight devices. A communicator that contains both the first eight and last eight devices is not supported. rankSize must be less than or equal to 8.
rankTableFile	string	-	Path of the configuration file for cluster information, which applies to single-node and multi-node communication scenarios. Currently, only the HCCL backend is supported. If a rank table is configured for a single-node system, the rank table is used to initialize the communicator.
commDomain	string	-	Communicator name used by a communication device group when there are multiple communicators. When the backend is lccl and commMode is multi-process, commDomain must be set to a value ranging from 0 to 65535. The memory size of the communicator can be set for performance optimization. The configuration format is commDomain:commDomain size (MB). For example, 0:400. When the configurations of multiple operators in the same communicator conflict, the configuration of the first operator executed in the communicator is used. Restriction: The default value is 200. Currently, the value cannot be less than 200.
rsv[64]	uint8_t	{0}	Reserved

Input

Parameter	Dimension	Data Type	Format	Description
x	[dim_0, dim_1, ..., dim_n]	hccl: float16/float/int8/int16/int32/int64/bf16 lccl: float16/float/int8/int16/int32/int64/bf16	ND	Input tensor

Constraints

rank, rankSize, and rankRoot must meet the following conditions:

0 ≤ rank < rankSize
0 ≤ rankRoot < rankSize

If there are multiple users, ATB_SHARE_MEMORY_NAME_SUFFIX (see ATB Environment Variables) needs to be used to distinguish the shared memory and synchronize initialization information.
When the communication operator of the ATB exits abnormally, run the following commands to clear the residual data to avoid affecting subsequent operations.
```
rm -rf /dev/shm/sem.lccl*
rm -rf /dev/shm/sem.hccl*
ipcrm -a
```
Currently, multiple HCCP process instances cannot run on a single device, and multiple communication operators cannot run on a single device at the same time. For example, when a communication operator is running, multiple models cannot run on the same card.

Parent topic: atb/infer_op_params.h