[object Object]

Note: This API will be deprecated in later versions. Use the latest aclnnGroupedMatmulV5 API instead.

[object Object][object Object]undefined[object Object]

• Description: Implements grouped matrix multiplication, supporting non-uniform matrix dimension sizes across multiple groups. The basic function is matrix multiplication, for example, $y_i[m_i,n_i]=x_i[m_i,k_i] \times weight_i[k_i,n_i], i=1...g$, where $g$ indicates the number of groups and $m_i$, $k_i$, and $n_i$ define the shapes for each group. Compared with , this API provides the following new features:
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • Supports axis grouping, represented by [object Object].
    • Supports the transposition of [object Object] and [object Object] in non-quantization scenarios. Transposition refers to the case where the shape is [M, K], the stride is [1, M], and the data layout is [K, M].
    • Supports FLOAT32 input for [object Object] and [object Object] in non-quantization scenarios.
    • Supports weight transposition and single-tensor weights in quantization and fake-quantization scenarios.
  • Ascend 950PR/Ascend 950DT:
    • Supports axis grouping, represented by [object Object].
    • Supports FLOAT32 input for [object Object] and [object Object] in non-quantization scenarios.
    • Supports weight transposition and single-tensor weights for [object Object], [object Object], and [object Object] in fake-quantization scenarios.
• Formula:
  • Non-quantization scenario:
  $$y_i=x_i\times weight_i + bias_i$$
  • Quantization scenario:
  $$y_i=(x_i\times weight_i + bias_i) * scale_i + offset_i$$
  • Dequantization scenario:
  $$y_i=(x_i\times weight_i + bias_i) * scale_i$$
  • Fake-quantization scenario:
  $$y_i=x_i\times (weight_i + antiquant\_offset_i) * antiquant\_scale_i + bias_i$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • x: FLOAT16, BFLOAT16, INT8, and FLOAT32
    • weight: FLOAT16, BFLOAT16, INT8, and FLOAT32
    • biasOptional: FLOAT16, FLOAT32, and INT32
    • y: FLOAT16, BFLOAT16, INT8, and FLOAT32
    • The input parameters x and weight, and the output parameter y support a maximum of 128 tensors.
  • Ascend 950PR/Ascend 950DT:
    • x: FLOAT16, BFLOAT16, and FLOAT32
    • weight: FLOAT16, BFLOAT16, FLOAT32, and INT8
    • biasOptional: FLOAT16, BFLOAT16, and FLOAT32
    • y: FLOAT16, BFLOAT16, and FLOAT32
    • scaleOptional and offsetOptional are not supported.
    • groupType supports grouping and non-grouping on the m axis. Only non-quantization supports grouping on the k axis.
    • In the non-quantization scenario, the input parameters x and weight, and the output parameter y support a maximum of 1024 tensors. In the fake-quantization scenario, the input parameters x and weight, and the output parameter y support a maximum of 128 tensors.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to deterministic implementation.

• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

  • The following input types are supported in non-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16
    • [object Object]: FLOAT32; [object Object]: FLOAT32; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: FLOAT32

  • The following input type is supported in quantization scenarios:

    • [object Object]: INT8; [object Object]: INT8; [object Object]: INT32; [object Object]: UINT64; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: INT8

  • The following input types are supported in fake-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: INT8; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: INT8; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: BFLOAT16
    • If [object Object] is passed, it must be a non-negative ascending array, and its length cannot be 1.

  • Supported scenarios for different [object Object] values:

    • In quantization and fake-quantization scenarios, [object Object] can be either [object Object] or [object Object].

    • "S" stands for single-tensor and "M" stands for multi-tensor, expressed in the sequence of [object Object], [object Object], [object Object]. For example, "SMS" indicates single-tensor [object Object], multi-tensor [object Object], and single-tensor [object Object].

      [object Object]undefined

  • The size of the last dimension for each tensor in [object Object] and [object Object] should be less than 65536. The last dimension of $x_i$ refers to the K-axis when [object Object] is false or the M-axis when [object Object] is true. The last dimension of $weight_i$ refers to the N-axis when [object Object] is false or the K-axis when [object Object] is true.

  • The size of each dimension for every tensor in [object Object] and [object Object], after 32-byte alignment, should be less than the maximum value of INT32 (2147483647).

• Ascend 950PR/Ascend 950DT:

  [object Object]
  • The following data types are supported in non-quantization scenarios:

    • If [object Object] is passed, it must be a non-negative ascending array, and its length cannot be 1.

    • The following input parameters are empty: scaleOptional, offsetOptional, antiquantScaleOptional, and antiquantOffsetOptional.

    • The data type combinations supported by the parameters that are not empty must meet the requirements in the following table.

      [object Object]undefined
  [object Object][object Object]
  • The following data types are supported in fake-quantization scenarios:

    • The following input parameters are empty: scaleOptional and offsetOptional.

    • The combinations of data types supported by non-empty parameters must meet the requirements listed in the following table.

      [object Object]undefined

    • The following table describes the requirements for antiquantScaleOptional, non-empty biasOptional, and antiquantOffsetOptional (g indicates the number of matmul groups, that is, the number of groups).

      [object Object]undefined

    • Only the SSS and MMM scenarios are supported.

  [object Object][object Object]

  • "S" stands for single-tensor and "M" stands for multi-tensor, expressed in the sequence of [object Object], [object Object], [object Object]. For example, "SMS" indicates single-tensor [object Object], multi-tensor [object Object], and single-tensor [object Object].

    [object Object]undefined
  [object Object]

[object Object]

The following example is for reference only. For details, see .