[object Object]

[object Object][object Object]undefined[object Object]

A fused operator of [object Object] and [object Object]. It performs a "combine" operation on the output of the [object Object] computation based on specified indices. [object Object] supports the AI processor-affinity format (NZ).

Compared with aclnnGroupedMatmulFinalizeRoutingWeightNz, this API has the following new features:

• The input parameters offsetOptional, antiquantScaleOptional, antiquantOffsetOptional, and tuningConfigOptional are added. The first three parameters are reserved and do not take effect currently. You can pass a null pointer.
• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The INT4 weight matrix is supported, and the tuningConfigOptional parameter is supported. The first value in the array indicates the expected number of tokens processed by each expert. During operator tiling, the operator is tiled based on the expected value, improving performance. Select the appropriate API as required.
• Ascend 950PR/Ascend 950DT: The per-token-per-channel and static per-tensor-per-channel quantization scenarios are added. For details, see [Quantization Overview](../common/Quantization Overview .md).

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  • x1 supports only INT8. The value range of dimension m is [1, 16 x 1024 x 8], and k supports 2048.
  • x2 supports INT4 and INT32. When the input is of type INT32, the dimension is (e, k, n / 8). When the input is converted to INT4, the dimension is (e, k, n). The value range of e is [1, 256], k supports 2048, and n supports 7168.
  • The shape of offsetOptional supports three dimensions, and the dimension is (e, 1, n). The values of e and n are the same as those of e and n in weight.
  • scaleOptional supports INT64, FLOAT32, and BF16.
  • rowIndex supports INT64 and INT32.
  • x1, x2, and groupListOptional are mandatory. scaleOptional, pertokenScaleOptional, logitOptional, rowIndexOptional, biasOptional, and sharedInputOptional are optional.

• Ascend 950PR/Ascend 950DT:

  • x1 supports the INT8, FLOAT8_E4M3FN and HIFLOAT8 data types.
  • x2 supports the INT8, FLOAT8_E4M3FN and HIFLOAT8 data types. The dimension is (e, k, n), and the value range of e is [1, 1024].
  • scaleOptional supports FLOAT32 and BF16.
  • When the data types of x1 and x2 are INT8, rowIndex supports the INT64 and INT32 data types. When the data types of x1 and x2 are FLOAT8_E4M3FN and HIFLOAT8, rowIndex supports the INT64 data type.
  • x1, x2, scaleOptional, groupListOptional, logitOptional, and rowIndexOptional are mandatory. pertokenScaleOptional, sharedInputOptional, and biasOptional are optional. Currently, the offsetOptional parameter is not supported.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following errors may be thrown:

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • For [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], aclnnGroupedMatmulFinalizeRoutingWeightNzV2 is implemented in non-deterministic mode by default. You can enable deterministic computing by using aclrtCtxSetSysParamOpt.
  • For Ascend 950PR/Ascend 950DT, aclnnGroupedMatmulFinalizeRoutingWeightNzV2 is implemented in non-deterministic mode by default. You cannot enable deterministic computing by using aclrtCtxSetSysParamOpt.

• For [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], the following data type combinations are supported for the input and output:

  [object Object]undefined

• Ascend 950PR/Ascend 950DT: The following data type combinations are supported for input and output.

  [object Object]undefined

[object Object]

The following example is for reference only. For details, see .

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  [object Object]

• Ascend 950PR/Ascend 950DT:

  [object Object]