Documentation
Download
[object Object]

[object Object][object Object]undefined
[object Object]

  • Description [object Object] extends the functionality of [object Object]. Building upon the support for FLOAT16/BFLOAT16 input types for x1 and x2:

    • Ascend 950PR/Ascend 950DT:
      • The low-precision data type FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8 is supported. It supports per-tensor, per-block, and mx [quantization methods](../common/quantization_introduction .md).
    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
      • Support for the low-precision data type INT8 is added. It supports per-token/per-channel .

  • Formulas:

    • Case 1: If the data types of x1 and x2 are FLOAT16 or BFLOAT16, the matmul operation is performed on the input parameters x1, x2, and bias, and then the ReduceScatter communication is performed.

      output=ReduceScatter(x1@x2+biasoptional)output=ReduceScatter(x1@x2 + bias_{optional})

    • Scenario 2: If the data type of x1 and x2 is FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8 in the per-tensor scenario, or INT8 in the per-channel or per-token scenario, and amaxOut is not output, x1 and x2 undergo MatMul and dequantization operations, followed by a ReduceScatter communication operation.

      output=ReduceScatter((x1Scale∗x2Scale)∗(x1@x2+biasoptional))output=ReduceScatter((x1Scale*x2Scale)*(x1@x2 + bias_{optional}))

    • Case 3: If the data types of x1 and x2 are FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8 in the perblock scenario, and amaxOut is not output, when x1 is (a0, a1) and x2 is (b0, b1), x1Scale is (ceildiv(a0, 128), ceildiv(a1, 128)) and x2Scale is (ceildiv(b0, 128), ceildiv(b1, 128)). After matmul and dequant are performed on the input parameters x1 and x2, the ReduceScatter communication is performed.

      output=ReduceScatter(∑0⌊kblockSize=128⌋(x1pr@x2rq∗(x1Scalepr∗x2Scalerq)))output=ReduceScatter(\sum_{0}^{\left \lfloor \frac{k}{blockSize=128} \right \rfloor} (x1_{pr}@x2_{rq}*(x1Scale_{pr}*x2Scale_{rq})))

    • Case 4: If the data types of x1 and x2 are FLOAT8_E4M3FN/FLOAT8_E5M2 in the mx quantization scenario, and amaxOut is not output, when x1 is (a0, a1) and x2 is (b0, b1), x1Scale is (a0, ceildiv(a1, 64), 2) and x2Scale is (b0, ceildiv(b1, 64), 2). After matmul and dequant are performed on the input parameters x1 and x2, the ReduceScatter communication is performed.

      output=ReduceScatter(∑0⌊kblockSize=32⌋(x1pr@x2rq∗(x1Scalepr∗x2Scalerq)))output=ReduceScatter(\sum_{0}^{\left \lfloor \frac{k}{blockSize=32} \right \rfloor} (x1_{pr}@x2_{rq}*(x1Scale_{pr}*x2Scale_{rq})))
[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation flow. Then, [object Object] is called to perform computation.

[object Object]
[object Object]
[object Object]

  • Parameters:

    [object Object]
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:
      • x1 and x2: When commMode is set to aiv, the data type can be FLOAT16, BFLOAT16, or INT8. The data format of x1 supports only ND, and the data format of x2 supports ND and FRACTAL_NZ.
      • bias: When commMode is set to aiv, only nullptr is supported in the current version.
      • x1Scale: When commMode is set to aiv, the data type can be FLOAT. When the data type of x1 and x2 is FLOAT16/BFLOAT16, only [object Object] is supported. In per-token scenarios, the shape is (m, 1).
      • x2Scale: When commMode is set to aiv, the data type can be FLOAT or INT64, and the data format is ND. The INT64 data type is only supported when the output data type is FLOAT16. When the data type of x1 and x2 is FLOAT16/BFLOAT16, only [object Object] is supported. In per-channel scenarios, the shape is (1, n).
      • groupSize: In the current version, only 0 is supported.
      • commMode: Currently, only the aiv mode is supported. In the [object Object] mode, the AI Vector Core is used to complete communication tasks. In the current version, only "aiv" is supported.
      • output: The data type can be FLOAT16 or BFLOAT16. If the data type of x1 is FLOAT16 or BFLOAT16, the data type of output is the same as that of x1.
    • Ascend 950PR/Ascend 950DT:
      • x1 and x2: The data type can be FLOAT16, BFLOAT16, FLOAT8_E4M3FN, FLOAT8_E5M2 or HIFLOAT8, and the data format is ND.
      • bias: If the data type of x1 is FLOAT16 or BFLOAT16, the data type of bias must be FLOAT16 or BFLOAT16. If the data type of x1 is FLOAT8_E4M3FN, FLOAT8_E5M2 or HIFLOAT8, the data type of bias must be FLOAT in the pertensor and mx quantization scenarios. In the perblock scenario, only nullptr is supported.
      • x1Scale: When the data types of x1 and x2 are FLOAT16 or BFLOAT16, the input can only be nullptr. In the pertensor scenario, the shape is [1]. In the perblock scenario, the shape is [ceildiv(m, 128), ceildiv(k, 128)]. In the pertensor and perblock scenarios, the data type can be FLOAT. In the mx quantization scenario, the data type is FLOAT8_E8M0 and the shape is (m, ceilDiv(k, 64), 2).
      • x2Scale: When the data types of x1 and x2 are FLOAT16 or BFLOAT16, the input can only be nullptr. In the pertensor scenario, the shape is [1]. In perblock mode, the shape is [ceildiv(k, 128), ceildiv(n, 128)]. In pertensor and perblock modes, the data type is FLOAT. In the mx scenario, the data type is FLOAT8_E8M0 and the shape is (n, ceilDiv(k, 64), 2).
      • groupSize: In perblock mode, when the x1Scale and x2Scale inputs are both 2D and the data type is FLOAT, the value of [groupSizeM, groupSizeN, groupSizeK] can only be [128, 128, 128], and the corresponding groupSize value is 549764202624. In the mx quantization scenario, when the x1Scale and x2Scale inputs are both 3D and the data type is FLOAT8_E8M0, the value of [groupSizeM, groupSizeN, groupSizeK] can only be [1, 1, 32], and the corresponding groupSize value is 4295032864. In other scenarios, only 0 is supported.
      • commMode: Only "ccu" is supported in the current version.
      • output: If the x1 type is FLOAT16 or BFLOAT16, the output type is the same as that of x1. If the x1 type is FLOAT8_E4M3FN, FLOAT8_E5M2 or HIFLOAT8, the data type can be FLOAT16, BFLOAT16, or FLOAT.groupSize=groupSizeK∣groupSizeN<<16∣groupSizeM<<32groupSize = groupSizeK | groupSizeN << 16 | groupSizeM << 32

  • Returns:

    [object Object]: status code. For details, see .

    The first-phase API implements input parameter validation. The following error codes may be returned.

    [object Object]
[object Object]

  • Parameters:

    [object Object]

  • Returns:

    [object Object]: status code. For details, see .

[object Object]

  • Deterministic compute:

    • By default, aclnnMatmulReduceScatterV2 uses deterministic computing.

  • Ascend 950PR/Ascend 950DT:

    • The x1 matrix cannot be transposed. The x2 matrix can be transposed or not transposed.
    • Input x1 must be 2D with shape (m, k). m must be an integer multiple of rank_size.
    • Input x2 must be 2D (k, n). The axes must meet the input parameter requirements of the [object Object] operator. The k axes must be equal and fall within the range of [256, 65535).
    • Input bias must be 1D (n,).
    • The output is 2D, and its shape is (m/rank_size, n), where rank_size indicates the number of devices.
    • When the data type of x1 and x2 is FLOAT16 or BFLOAT16, x1 and x2 support empty tensors. In this case, m and n can be empty, but k cannot be empty and must meet the following conditions:
      • m is empty, k is not empty, and n is not empty.
      • If m is not empty, k is not empty, and n is empty.
      • If m is empty, k is not empty, and n is empty.
    • When the data type of x1 and x2 is FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8, empty tensors are not supported.
    • When the data type of x1 and x2 is FLOAT16, BFLOAT16, or HIFLOAT8, the data types of x1 and x2 must be the same.
    • When the data type of x1 and x2 is FLOAT8_E4M3FN/FLOAT8_E5M2, the data of x1 and x2 can be either of them.
    • 2, 4, 8, 16, 32, or 64 cards are supported.
    • The total size of ReduceScatter collective communication data cannot exceed 16256 MB. The total size of collective communication data is calculated as follows: m x n x sizeof(output_dtype). The internal implementation of the operator may vary according to the shape. The actual supported total communication volume may be slightly less than this value.

  • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object] and [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

    • The x1 matrix cannot be transposed. The x2 matrix can be transposed or not transposed.
    • Input x1 must be 2D with shape (m, k). m must be an integer multiple of rank_size.
    • Input x2 must be 2D (k, n). The axes must meet the input parameter requirements of the [object Object] operator. The k axes must be equal and fall within the range of [256, 65535).
    • Input bias must be 1D (n,).
    • The output is 2D, and its shape is (m/rank_size, n), where rank_size indicates the number of devices.
    • Empty tensors are not supported.
    • The data types of x1 and x2 must be the same.
    • Two, four, and eight devices are supported.
[object Object]

The following example is for reference only. For details, see .

Note: This sample code calls some HCCL collective communication library APIs: HcclGetCommName, HcclCommInitAll, and HcclCommDestroy. For details, see .

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

    [object Object]

  • Ascend 950PR/Ascend 950DT:

    [object Object]
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details