[object Object]

Note: This API will be deprecated in later versions. Use aclnnQuantMatmulAllReduceV2 and aclnnAddRmsNorm instead.

[object Object][object Object]undefined

Note: When using this API, ensure that the driver firmware package and CANN package are in the 8.0.RC2 version or later. Otherwise, an error, such as BUS ERROR, will be reported.

[object Object]

• Description: Performs the MatMul, AllReduce, addition, and RMSNorm computation in sequence.
• Formulas:$$mm_out = allReduce(dequantScale * (x1_{int8}@x2_{int8} + bias_{int32}))$$ $$y = mm_out + residual$$ $$normOut = \frac{y}{RMS(y)} * gamma, RMS(y) = \sqrt{\frac{1}{d} \sum_{i=1}^{d} y_{i}^{2} + epsilon}$$

[object Object]

• [object Object] and [object Object] implement the same function in different ways. Select a proper operator based on your requirements.

  • [object Object]: Output tensor objects [object Object] and [object Object] need to be created to store the computation result.
  • [object Object]: Output tensor object [object Object] needs to be created. The results that would have been stored in the output tensor [object Object] in the non-inplace scenario are directly written to the memory of the input tensor [object Object].

• Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computing:

  • [object Object] defaults to non-deterministic implementation. You can set the HCCL_DETERMINISTIC environment variable to true to enable deterministic computation.

• The application scenario is the same as that of aclnnQuantMatmulAllReduce. MC2 is disabled in incremental generation scenarios but enabled in full generation scenarios

• [object Object] can be 2D (m, k) or 3D (b, s, k). [object Object] must be 2D (k, n), where the axes meet the input parameter requirements of the MatMul operator. The k axes of [object Object] and [object Object] must be equal. If [object Object] is not empty, it must be 1D with shape (n).

• The value of m cannot exceed 2147483647. The size of the last dimension of [object Object] (k) and [object Object] (k when transposed and n when not transposed) cannot exceed 65535.

• The input [object Object] must be 3D (b, s, n). When [object Object] is 2D, (b × s) of [object Object] equals m of [object Object]. The shape of the input [object Object] is (n).

• The dimensions and data types of the outputs [object Object] and [object Object] are the same as those of [object Object], with shape (b, s, n).

• If the output [object Object] is of FLOAT16 type, the type of [object Object] is UINT64 or INT64. If the output [object Object] is of BFLOAT16 type, the type of [object Object] is BFLOAT16.

• The data type of [object Object] and [object Object] is INT8, and the data type of [object Object] is INT32. The data types of [object Object], [object Object], [object Object], and [object Object] must be the same.

• The [object Object] matrix cannot be transposed. The [object Object] matrix can be transposed or not transposed.

• 1, 2, 4, and 8 ranks are supported, and only the all-mesh networking of HCCS links is supported.

• Empty tensors with (b × s) and n of 0 are supported. Empty tensors with k of 0 are not supported.

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: supports only one communication domain for MC2 operators within a model.

[object Object]

The following example is for reference only. For details, see .