Note: When using this API, ensure that the driver firmware package and CANN package are in the 8.0.RC2 version or later. Otherwise, an error, such as BUS ERROR, will be reported.
API function: Implements AllReduce communication of low-bit data, dequantizes data during communication, and outputs communication results. There are two cases based on the data volume:
Formula:
The Reduce computation is performed on data from different ranks.
This operator has calls. You must call the aclnnQuantAllReduceGetWorkspaceSize API to obtain the workspace size required for computation and the executor that contains the operator computation process, and then call the aclnnQuantAllReduce API to perform computation.
Parameters
[object Object]Returns:
[object Object]: status code. For details, see .The first-phase API implements input parameter verification. The following errors may be thrown.
[object Object]
Deterministic computation:
- By default, aclnnQuantAllReduce is implemented in a non-deterministic manner. You can enable deterministic computation by using aclrtCtxSetSysParamOpt.
When the data type of x is FLOAT8_E4M3FN or FLOAT8_E5M2 and the data type of scales is FLOAT8_E8M0, the input data is quantized in mx mode.
When the data type of x is INT8, HIFLOAT8, FLOAT8_E4M3FN or FLOAT8_E5M2 and the data type of scales is FLOAT, the input data is quantized in pertoken-pergroup mode (groupSize = 128).
This feature is enabled only on the Ascend 950 series platforms.
Empty tensor input is not supported.
The size of the communicator can be 2, 4, or 8.
Restrictions on the use of the communicator: The aclnnQuantAllReduce and aclnnQuantReduceScatter operators can be executed only in sequence in the same communicator, and other communication operators are not allowed in the communicator.
HCCL_BUFFSIZE: Before calling this operator, check whether the value of the HCCL_BUFFSIZE environment variable is proper. This environment variable indicates the buffer size occupied by a single communication domain, in MB. If this environment variable is not set, the default value 200 MB is used. The following condition must be met: HCCL_BUFFSIZE >= 2 (xDataSize + scalesDataSize + 1). xDataSize indicates the size of the input x data, which is calculated as follows: xDataSize = b s H 1 (byte). scalesDataSize indicates the size of the scales data. When the quantization mode is per-token per-group quantization, the calculation formula is as follows: scalesDataSize = b s H / 128 4 (byte). When the quantization mode is mx quantization, the calculation formula is as follows: scalesDataSize = b s H / 32 1 (byte).
The value of H must be 128-aligned and within the range of [1024, 8192].
The following example is for reference only. For details, see .
Note: This sample code calls some HCCL collective communication library APIs, including HcclCommInitClusterInfoConfig, HcclGetCommName, and HcclCommDestroy. For details, see .
Ascend 950PR/Ascend 950DT:
[object Object]