[object Object][object Object][object Object]undefined

Note: When using this API, ensure that the driver firmware package and CANN package are in the 8.0.RC2 version or later. Otherwise, an error, such as BUS ERROR, will be reported.

[object Object]

• Description: Performs mm + all_reduce + add + rms_norm computation.
• Formula:$$mm\_out = allReduce(x1 @ (x2*antiquantScale + antiquantOffset) + bias)$$ $$y = mm\_out + residual$$ $$normOut = \frac{y}{RMS(y)} * gamma, RMS(y) = \sqrt{\frac{1}{d} \sum_{i=1}^{d} y_{i}^{2} + epsilon}$$

[object Object]

• [object Object] and [object Object] implement the same function in different ways. Select a proper operator based on your requirements.

  • [object Object]: Output tensor objects [object Object] and [object Object] need to be created to store the computation result.
  • [object Object]: Output tensor [object Object] needs to be created. The results that would have been stored in the output tensor [object Object] in the original non-inplace scenario are directly written to the memory of the input tensor [object Object].

• Each operator has calls. First call [object Object] to obtain the required workspace size for computation and the executor that includes the operator's computation process. Then, call [object Object] to execute the computation.

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to a non-deterministic implementation. You can call [object Object] to enable deterministic computation.

• The application scenario is the same as that of [object Object]. MC2 is disabled in incremental scenarios but enabled in full scenarios.

• The input [object Object] can be two-dimensional or three-dimensional, with shape (b, s, k) or (s, k), respectively. [object Object] must be two-dimensional, with a shape of (k, n), and the axes must meet the requirements of the mm operator input parameters, with the k axis being equal. The range of b*s and s is [1, 2147483647], and the range of k and n is [1, 65535]. If bias is not empty, bias is one-dimensional, with a shape of (n).

• The input [object Object] must be three-dimensional, with a shape of (b, s, n). When [object Object] is two-dimensional, the (b*s) of [object Object] equals the [object Object] of [object Object]. The input [object Object] must be one-dimensional, with a shape of (n).

• The [object Object] satisfies the following scenarios: for [object Object], the shape is [object Object]; for [object Object], the shape is [object Object] or [object Object]; for [object Object], the shape is [object Object]. If [object Object] is not empty, its shape is consistent with [object Object].

• The dimensions and data types of the output [object Object] and [object Object] are the same as [object Object]. If [object Object] is not empty, its shape size is equal to the last dimension of [object Object].

• The data type of [object Object] must be INT8 or INT4, and the data types of [object Object], [object Object], [object Object], [object Object], [object Object], and [object Object] computation inputs must be consistent.

• Only supports [object Object] matrix transpose/non-transpose, [object Object] matrix supports non-transpose scenarios.

• The value of antiquantGroupSize should satisfy the range [32, min(k-1, INT_MAX)] and be a multiple of 32.

• Supports ranks 1, 2, 4, and 8, and only supports full-mesh networking with HCCS links.

• Supports empty tensors with (b*s) and n as 0, but does not support empty tensors with k as 0.

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: Only the same communication domain for MC2 operators within a model is supported.

[object Object]

The following example is for reference only. For details, see .