[object Object][object Object][object Object]undefined

Note: When using this API, ensure that the driver firmware package and CANN package are in the 8.0.RC2 version or later. Otherwise, an error, such as BUS ERROR, will be reported.

[object Object]

• Interface Function: Perform pseudo-quantization computation on the input parameter x2, then complete the Matmul and AllReduce computation. Supports pertensor, perchannel, and pergroup quantization methods.

• Formula:

  $$output = allreduce(x1 @ ((x2 + antiquantOffset) *antiquantScale) + bias+ x3)$$

[object Object]

Each operator has calls. First call [object Object] to obtain the required workspace size for computation and the executor that includes the operator's computation process. Then, call [object Object] to perform the computation.

[object Object]

• Parameters:

  [object Object]
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • The data types supported for input x2 are INT8 and INT4, and the data formats supported are ND (currently only 2D input is supported) and FRACTAL_NZ (currently only 4D input is supported). When the data format of x2 is FRACTAL_NZ, use [object Object] and [object Object] to complete the conversion from ND to NZ input. Non-contiguous tensors are only supported in transpose scenarios.
    • The data type of the input bias should be consistent with that of x1.
    • The data type of input x3 supports BFLOAT16 and FLOAT16.
    • The data type of the output supports BFLOAT16 and FLOAT16.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters:

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to a non-deterministic implementation. You can call [object Object] to enable deterministic computation.

• MC2 is disabled in incremental scenarios but enabled in full scenarios.

• The input x1 can be two-dimensional or three-dimensional, with a shape of (b, s, k) or (m, k).

• x2 must be two-dimensional. Its shape is (k, n), where the k-axis meets the input requirements of the mm operator, the k-axis is equal, the range of m is [1, 2147483647], and the ranges of k and n are [1, 65535].

• The passed x1, x2, antiQuantScale, or output is not a null pointer.

• When the shape of input x1 is (b, s, k), the shape of x3 (non-empty scenario) and the output is (b, s, n); when the shape of input x1 is (m, k), the shape of x3 (non-empty scenario) and the output is (m, n).

• If bias is not empty, its shape size is equal to the size of the last dimension of the output. In the pertensor scenario, the shape of antiQuantScale is (1); in the perchannel scenario, the shape is (1,n)/(n); in the pergroup scenario, the shape is (ceil(k,antiQuantGroupSize), n). If antiQuantOffset is not empty, its shape is consistent with antiQuantScale.

• The data types and formats of x1, x2, x3 (non-empty scenarios), antiquantScale, antiquantOffset (non-empty scenarios), output, and bias (non-empty scenarios) must be within the supported range.

• x1, antiquantScale, antiquantOffset (non-empty scenario), x3 (non-empty scenario), bias (non-empty scenario) have the same data type for output. The value of antiquantGroupSize satisfies the range and is a multiple of 32.

• In the per-group scenario, when transposing x2, both the antiquantScale and antiquantOffset need to be transposed together to maintain continuity.

• In long sequence scenarios, as b/s or m increases, OOM or computation timeout may occur.

• Only supports all mesh networking for hccs links.

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: Ranks 1, 2, 4, and 8 are supported.

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: Only the same communication domain for MC2 operators within a model is supported.

[object Object]

The following example is for reference only. For details, see .

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

  [object Object]