[object Object][object Object][object Object]undefined[object Object]

• API function: [object Object] extends the functionality of [object Object]. Building upon the support for FLOAT16/BFLOAT16 input types for [object Object] and [object Object]:

  • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • Support for the low-precision data type INT8 is added. It supports per-token/per-channel .
    • Support for FLOAT8_E4M3FN/FLOAT8_E5M2 mixfp8 quantization is added.

• Formula:

  • Scenario 1: If the data type of [object Object] and [object Object] is FLOAT16/BFLOAT16, the input [object Object] and [object Object] undergo a MatMul operation, followed by a [object Object] communication operation.
  $$output=ReduceScatter(x1@x2)$$
  • Scenario 2: If the data type of [object Object] and [object Object] is FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8 in the per-tensor scenario, or INT8 in the per-channel or per-token scenario, and [object Object] is not output, [object Object] and [object Object]2 undergo MatMul and dequantization operations, followed by a [object Object] communication operation.
  $$output=ReduceScatter((x1Scale*x2Scale)*(x1@x2 + bias_{optional}))$$
  • Scenario 3: If the data type of [object Object] and [object Object] is FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8 in the per-block scenario, and amaxOut is not output, when [object Object] is (a0, a1) and [object Object] is (b0, b1), and x1Scale is (ceildiv(a0, 128), ceildiv(a1, 128)) and x2Scale is (ceildiv(b0, 128), ceildiv(b1, 128)), [object Object] and [object Object] undergo MatMul and dequantization operations, followed by a [object Object] communication operation.
  $$output=ReduceScatter(\sum_{0}^{\left \lfloor \frac{k}{blockSize} \right \rfloor} (x1_{pr}@x2_{rq}*(x1Scale_{pr}*x2Scale_{rq})))$$
  • Scenario 4: If the data type of [object Object] and [object Object] is FLOAT8_E4M3FN/FLOAT8_E5M2 in the per-tensor mixed FP8 quantization scenario, and amaxOut is not output, when [object Object] is (a0, a1, 2) and [object Object] is (b0, b1, 2), and x1Scale is (a0, ceildiv(a1, 64), 2) and x2Scale is (b1, ceildiv(b0, 64), 2), with [object Object] not transposed and [object Object] transposed, [object Object] and [object Object] undergo MatMul and dequantization operations, followed by a [object Object] communication operation.
  $$output=ReduceScatter(\sum_{0}^{\left \lfloor \frac{k}{blockSize=32} \right \rfloor} (x1_{pr}@x2_{rq}*(x1Scale_{pr}*x2Scale_{rq})))$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters

  • [object Object] (aclTensor*, input): two-dimensional aclTensor on the device, representing the left matrix for [object Object],
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: When [object Object] is [object Object], supported data types include FLOAT16 and BFLOAT16. When [object Object] is [object Object], supported data types include FLOAT16, BFLOAT16, and INT8. The shape is [m, k]. The is ND. The current version only supports two-dimensional input shapes and non-transposed scenarios.
  • [object Object] (aclTensor*, input): two-dimensional aclTensor on the device, representing the left matrix for [object Object],
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: When [object Object] is [object Object], supported data types include FLOAT16 and BFLOAT16. When [object Object] is [object Object], supported data types include FLOAT16, BFLOAT16, and INT8. The shape is [k, n]. The is ND. [Non-contiguous tensors]](../common/non-contiguous_tensor.md) can be constructed via transposition. The current version only supports two-dimensional input shapes.
  • [object Object] (aclTensor*, input): one-dimensional aclTensor on the device.
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: When [object Object] is [object Object], supported data types include FLOAT16 and BFLOAT16, and the is ND. The current version supports only 0 as input. When [object Object] is [object Object], the current version only supports [object Object] as input.
  • [object Object] (aclTensor*, input): aclTensor on the device, representing the dequantization parameter for the left [object Object] matrix.
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: When [object Object] is [object Object], only [object Object] is supported. When [object Object] is [object Object], the data type is FLOAT, and the is ND. When the data type of [object Object] and [object Object] is FLOAT16/BFLOAT16, only [object Object] is supported. In per-token scenarios, the shape is (m, 1).
  • [object Object] (aclTensor*, input): aclTensor on the device, representing the dequantization parameter for the right [object Object] matrix.
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: When [object Object] is [object Object], only [object Object] is supported. When [object Object] is [object Object], the data type is FLOAT, INT64, and the data format is ND. The INT64 data type is only supported when the output data type is FLOAT16. When the data type of [object Object] and [object Object] is FLOAT16/BFLOAT16, only [object Object] is supported. In per-channel scenarios, the shape is [object Object].
  • [object Object] (aclTensor*, input): one-dimensional aclTensor on the device, representing the quantization parameter for the [object Object] output matrix. The data type can be FLOAT. The current version only supports [object Object] as input.
  • [object Object] (int64_t, input): integer on the host used to indicate the number of elements along the M and N axes of the [object Object] output matrix that corresponds to a quantization parameter in that direction. [object Object] is composed of [object Object], [object Object], and [object Object]. Each value occupies 16 bits, and the formula is [object Object]. The [object Object] output matrix does not involve the K axis, therefore, [object Object] is fixed at 0. The current version supports only [object Object] for blockSizeM and blockSizeN.
  • [object Object] (char*, input): character string that identifies a column group on the host. The data type is string. It is obtained through [object Object]. [object Object] is the group.
  • [object Object] (char*, input): char on the host, reduce operation type. The data type is string. The current version only supports [object Object] as input.
  • [object Object] (int64_t, input): integer on the host representing the number of splits communication data copies, equal to the value of total data volume divided by single communication volume. The data type can be INT64. The current version supports only [object Object] as input.
  • [object Object] (int64_t, input): integer on the host, indicating the enumeration of the AscendCL stream mode. Currently, only the enumerated value 1 is supported. The data type can be INT64.
  • [object Object] (int64_t, Input): number of elements in the [object Object] or [object Object] input along a specific dimension that a single value in [object Object] or [object Object] applies to during dequantization. [object Object] is composed of [object Object], [object Object], and [object Object]. Each value occupies 16 bits, and the formula is [object Object].
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The current version supports only 0 as input.
  • [object Object] (char*, input): integer on the host representing the communication mode.
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: Currently, the [object Object] and [object Object] modes are supported. In the [object Object] mode, the AI CPU is used to complete communication, which is equivalent to the [object Object] operator. In the [object Object] mode, the AI Vector Core is used to complete communication tasks.
  • [object Object] (aclTensor*, output): aclTensor on the device representing the result of MatMul computation and [object Object] communication.
    • For [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: Supported data types include FLOAT16 and BFLOAT16, and the data type is ND. If the [object Object] data type is FLOAT16 or BFLOAT16, the [object Object] data type is the same as that of [object Object].
  • [object Object] (aclTensor*, output): one-dimensional aclTensor on the device representing the maximum value result of MatMul computation. The current version only supports [object Object] or an empty tensor.
  • [object Object] (uint64_t*, output): integer on the device representing the size of the workspace required to be allocated on the device.
  • [object Object] (aclOpExecutor**, output): aclOpExecutor on the device that returns the operator executor, containing the operator computation process.

• Returns

  [object Object] status code. For details, see .

  [object Object]

[object Object]

• Parameters

  • [object Object] (void*, input): address of the workspace to be allocated on the device.
  • [object Object] (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API [object Object].
  • [object Object] (aclOpExecutor*, input): operator executor, containing the operator computation process.
  • [object Object] (aclrtStream, input): stream for executing the task.

• Returns

  [object Object] status code. For details, see .

[object Object]

• Deterministic compute:
  • [object Object] defaults to a non-deterministic implementation. You can call [object Object] to enable deterministic compute.
• [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
  • The [object Object] matrix cannot be transposed. The [object Object] matrix can be transposed or not transposed.
  • Input [object Object] must be 2D with shape [object Object]. m must be an integer multiple of [object Object].
  • Input [object Object] must be 2D with shape [object Object]. The axes must meet the input parameter requirements of the [object Object] operator. The k axes must be equal and fall within the range of [256, 65535).
  • Input bias must be 1D [object Object].
  • The output is 2D, and its shape is (m/rank_size, n), where [object Object] indicates the number of devices.
  • Empty tensors are not supported.
  • The data types of [object Object] and [object Object] must be the same.
  • Two, four, and eight devices are supported.

[object Object]

The following example is for reference only. For details, see .