[object Object][object Object][object Object]undefined[object Object]

• Description: Implements grouped matrix multiplication, supporting non-uniform matrix dimension sizes across multiple groups. The basic function is matrix multiplication, for example, $y_i[m_i,n_i]=x_i[m_i,k_i] \times weight_i[k_i,n_i], i=1...g$, where $g$ indicates the number of groups and $m_i$, $k_i$, and $n_i$ define the shapes for each group.

  Compared with , this API provides the following new features:

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
    • Supports axis grouping, represented by [object Object].
    • Supports the transposition of [object Object] and [object Object] in non-quantization scenarios. Transposition refers to the case where the shape is [M, K], the stride is [1, M], and the data layout is [K, M].
    • Supports FLOAT32 input for [object Object] and [object Object] in non-quantization scenarios.
    • Supports weight transposition and single-tensor weights in quantization and fake-quantization scenarios.

• Formula:

  • Non-quantization scenario:
  $$y_i=x_i\times weight_i + bias_i$$
  • Quantization scenario:
  $$y_i=(x_i\times weight_i + bias_i) * scale_i + offset_i$$
  • Dequantization scenario:
  $$y_i=(x_i\times weight_i + bias_i) * scale_i$$
  • Fake-quantization scenario:
  $$y_i=x_i\times (weight_i + antiquant\_offset_i) * antiquant\_scale_i + bias_i$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters

  • x (aclTensorList*, computation input): required parameter, aclTensorList on the device, $x$ in the formula. The can be ND, and the maximum length is 128.
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The data type can be FLOAT16, BFLOAT16, INT8, or FLOAT32.
  • weight (aclTensorList*, computation input): required parameter, aclTensorList on the device, $weight$ in the formula. The can be ND, and the maximum length is 128.
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The data type can be FLOAT16, BFLOAT16, INT8, or FLOAT32.
  • biasOptional (aclTensorList*, computation input): optional parameter, aclTensorList on the device, $bias$ in the formula. The can be ND, and the length is the same as that of [object Object].
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The data type can be FLOAT16, FLOAT32, or INT32.
  • scaleOptional (aclTensorList*, computation input): optional parameter, aclTensorList on the device, indicating the scale factor for quantization parameters. The can be ND, and the length is the same as that of [object Object].
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The data type can be UINT64.
  • offsetOptional (aclTensorList*, computation input): optional parameter, aclTensorList on the device, indicating the offset for quantization parameters. The can be ND, and the length is the same as that of [object Object].
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The data type can be FLOAT32.
  • antiquantScaleOptional (aclTensorList*, computation input): optional parameter, aclTensorList on the device, indicating the scale factor for fake-quantization parameters. The can be ND, the length is the same as that of [object Object], and the data type can be FLOAT16 or BFLOAT16.
  • antiquantOffsetOptional (aclTensorList*, computation input): optional parameter, aclTensorList on the device, indicating the offset for fake-quantization parameters. The can be ND, the length is the same as that of [object Object], and the data type can be FLOAT16 or BFLOAT16.
  • groupListOptional (aclIntArray*, computation input): optional parameter, aclIntArray type on the host, indicating the Matmul indexing along the grouping axis represented by [object Object]. The data type can be INT64, the can be ND, and the length is the same as that of [object Object]. Note that when the length of the TensorList in the output is 1, the last value in [object Object] constrains the valid portion of the output data. Any portion not specified in [object Object] will not be updated.
  • splitItem (int64_t, computation input): integer type, indicating whether tensor splitting is required for the output. [object Object] or [object Object] indicates multi-tensor, and [object Object] or [object Object] indicates single-tensor.
  • groupType (int64_t, computation input): integer type, indicating the axis to be grouped. For example, if the matrix multiplication is [object Object], [object Object] has the following options: [object Object] means no axis grouping (default), [object Object] indicates M-axis grouping, [object Object] indicates N-axis grouping (not supported currently), and [object Object] indicates K-axis grouping.
  • y (aclTensorList*, computation output): aclTensorList on the device, $y$ in the formula. The can be ND, and the maximum length is 128.
    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The data type can be FLOAT16, BFLOAT16, INT8, or FLOAT32.
  • workspaceSize (uint64_t*, output): size of the workspace to be allocated on the device.
  • executor (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Return

  [object Object] status code. For details, see .

[object Object]

• Parameters

  • workspace (void*, input): address of the workspace to be allocated on the device.
  • workspaceSize (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling [object Object].
  • executor (aclOpExecutor*, input): operator executor, containing the operator computation process.
  • stream (aclrtStream, input): stream for executing the task.

• Return

  [object Object] status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to deterministic implementation.
• [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:

  • The following input types are supported in non-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16
    • [object Object]: FLOAT32; [object Object]: FLOAT32; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: FLOAT32

  • The following input types are supported in quantization scenarios:

    • [object Object]: INT8; [object Object]: INT8; [object Object]: INT32; [object Object]: UINT64; [object Object]: null; [object Object]: null; [object Object]: null; [object Object]: INT8

  • The following input types are supported in fake-quantization scenarios:

    • [object Object]: FLOAT16; [object Object]: INT8; [object Object]: FLOAT16; [object Object]: null; [object Object]: null; [object Object]: FLOAT16; [object Object]: FLOAT16; [object Object]: FLOAT16
    • [object Object]: BFLOAT16; [object Object]: INT8; [object Object]: FLOAT32; [object Object]: null; [object Object]: null; [object Object]: BFLOAT16; [object Object]: BFLOAT16; [object Object]: BFLOAT16
    • If [object Object] is passed, it must be a non-negative ascending array, and its length cannot be 1.

  • Supported scenarios for different [object Object] values:

    • In quantization and fake-quantization scenarios, [object Object] can be either [object Object] or [object Object].
    • "S" stands for single-tensor and "M" stands for multi-tensor, expressed in the sequence of [object Object], [object Object], [object Object]. For example, "SMS" indicates single-tensor [object Object], multi-tensor [object Object], and single-tensor [object Object].
    [object Object]undefined

  • The size of the last dimension for each tensor in [object Object] and [object Object] should be less than 65536. The last dimension of $x_i$ refers to the K-axis when [object Object] is false or the M-axis when [object Object] is true. The last dimension of $weight_i$ refers to the N-axis when [object Object] is false or the K-axis when [object Object] is true.

  • The size of each dimension for every tensor in [object Object] and [object Object], after 32-byte alignment, should be less than the maximum value of INT32 (2147483647).

[object Object]

The following example is for reference only. For details, see .