Documentation
Download
[object Object]

[object Object][object Object]undefined
[object Object]

  • Description: Performs matrix multiplication for quantization.

    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

      It is compatible with the aclnnQuantMatmulV3 and aclnnQuantMatmulV4 APIs. Performs quantized matrix multiplication. The minimum input dimension is 1 and the maximum input dimension is 2. Similar APIs include aclnnMm (only two-dimensional tensors can be used as the input of matrix multiplication).

    • Ascend 950PR/Ascend 950DT:

      Compatible with the aclnnQuantMatmulV3 and aclnnQuantMatmulV4 APIs. In addition to the functions of the two APIs, this API supports the G-B, B-B, T-CG, and mx quantization modes (../common/Quantization.md), and the x1 and x2 inputs support the FLOAT8_E4M3FN, FLOAT8_E5M2, HIFLOAT8 and FLOAT4_E2M1 data types. Performs quantized matrix multiplication, supporting at least two-dimensional input and at most six-dimensional-dimensional input. Similar APIs include aclnnMm (only two-dimensional tensors can be used as the input of matrix multiplication) and aclnnBatchMatMul (only three-dimensional matrix multiplication is supported, whose first dimension is the batch dimension).

  • Formula:

    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

      K-C && K-T, T-C && T-T, G-B and K-G quantization modes are supported. For details about the input and output data type combinations corresponding to different quantization modes, see .

      [object Object][object Object]

      • x1 is of type INT8, x2 is of type INT32, x1Scale is of type FLOAT32, x2Scale is of type UINT64 or INT64, and yOffset is of type FLOAT32:

        out=((x1@(x2∗x2Scale))+yOffset)∗x1Scaleout = ((x1 @ (x2*x2Scale)) + yOffset) * x1Scale

      • x1 and x2 are of type INT4, x1Scale and x2Scale are of type FLOAT32, x2Offset is of type FLOAT16, and out is of type FLOAT16 or BFLOAT16 (asymmetric quantization per token per group):

        out=x1Scale∗x2Scale∗(x1@x2−x1@x2Offset)out = x1Scale * x2Scale * (x1 @ x2 - x1 @ x2Offset)
      [object Object][object Object][object Object]

      • With x1Scale, no bias:

        out=x1@x2∗x2Scale∗x1Scaleout = x1@x2 * x2Scale * x1Scale

      • x1Scale and bias are of type INT32 (no offset in this scenario):

        out=(x1@x2+bias)∗x2Scale∗x1Scaleout = (x1@x2 + bias) * x2Scale * x1Scale

      • x1Scale and bias are of type BFLOAT16, FLOAT16, or FLOAT32 (no offset in this scenario):

        out=x1@x2∗x2Scale∗x1Scale+biasout = x1@x2 * x2Scale * x1Scale + bias
      [object Object][object Object][object Object]

      • No x1Scale, no bias:

        out=x1@x2∗x2Scale+x2Offsetout = x1@x2 * x2Scale + x2Offset

      • bias (INT32):

        out=(x1@x2+bias)∗x2Scale+x2Offsetout = (x1@x2 + bias) * x2Scale + x2Offset

      • bias (BFLOAT16/FLOAT32) (no offset in this scenario):

        out=x1@x2∗x2Scale+biasout = x1@x2 * x2Scale + bias
      [object Object][object Object][object Object]
      • x1 and x2 are of type INT8, x1Scale and x2Scale are of type FLOAT32, bias is of type FLOAT32, and out is of type FLOAT16 or BFLOAT16 (pergroup-perblock quantization):out=(x1@x2)∗x1Scale∗x2Scale+biasout = (x1 @ x2) * x1Scale * x2Scale + bias
      [object Object]

    • [object Object]Atlas inference products[object Object]:

      K-C is supported. For details about the input and output data type combinations corresponding to different quantization modes, see .

      [object Object][object Object]

      • With x1Scale, no bias:

        out=x1@x2∗x2Scale∗x1Scaleout = x1@x2 * x2Scale * x1Scale

      • x1Scale and bias of type INT32 (no offset in this scenario):

        out=(x1@x2+bias)∗x2Scale∗x1Scaleout = (x1@x2 + bias) * x2Scale * x1Scale
      [object Object]

    • Ascend 950PR/Ascend 950DT:

      T-C && T-T, K-C && K-T, G-B , B-B , mx and T-CG quantization modes are supported. For details about the input and output data type combinations corresponding to different quantization modes, see .

      [object Object][object Object]

      • x1 and x2 are of type int8. x1Scale is not supported. x2Scale is of type int64 or uint64. The optional parameter x2Offset is of type float32, and the optional parameter bias is of type int32.

        out=(x1@x2+bias)∗x2Scale+x2Offsetout = (x1@x2 + bias) * x2Scale + x2Offset

      • x1 and x2 are of type int8. x1Scale is not supported. x2Scale is of type int64 or uint64. The optional parameter bias is of type int32. x1 and x2 are of type FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8. x1Scale is not supported. x2Scale is of type int64 or uint64. The optional parameter bias is of type float32.

        out=(x1@x2+bias)∗x2Scaleout = (x1@x2 + bias) * x2Scale

      • x1 and x2 are of type int8. x1Scale is not supported. x2Scale is of type bfloat16 or float32. The optional parameter bias is of type bfloat16 or float32.

        out=x1@x2∗x2Scale+biasout = x1@x2 * x2Scale + bias

      • x1 and x2 are of type FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8. x1Scale is of type float32. x2Scale is of type float32. The optional parameter bias is of type float32.

        out=(x1@x2+bias)∗x2Scale∗x1Scaleout = (x1@x2 + bias) * x2Scale * x1Scale
      [object Object][object Object][object Object]

      • x1 and x2 are of type int8. x1Scale is of type float32. x2Scale is of type bfloat16 or float32. The optional parameter bias is of type int32. or x1 and x2 are of type FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8, x1Scale is of type FLOAT32, x2Scale is of type FLOAT32, and the optional bias parameter is of type FLOAT32:

        out=(x1@x2+bias)∗x2Scale∗x1Scaleout = (x1@x2 + bias) * x2Scale * x1Scale

      • x1 and x2 are of type INT8, x1Scale is of type FLOAT32, x2Scale is of type BFLOAT16 or FLOAT32, and the optional bias parameter is of type BFLOAT16 or FLOAT32. or x1 and x2 are of type INT8, x1Scale is of type FLOAT32, x2Scale is of type FLOAT32, and the optional bias parameter is of type FLOAT16 or FLOAT32:

        out=x1@x2∗x2Scale∗x1Scale+biasout = x1@x2 * x2Scale * x1Scale + bias
      [object Object][object Object][object Object]out[m,n]=∑j=0kLoops−1((∑k=0gsK−1(x1Slice∗x2Slice))∗(x1Scale[m/gsM,j]∗x2Scale[j,n/gsN]))+bias[n]out[m,n] = \sum_{j=0}^{kLoops-1} ((\sum_{k=0}^{gsK-1} (x1Slice * x2Slice))* (x1Scale[m/gsM, j] * x2Scale[j, n/gsN]))+bias[n]

      gsM, gsN, and gsK represent groupSizeM, groupSizeN, and groupSizeK, respectively. x1Slice represents the vector of length groupSizeK in the mth row of x1, and x2Slice represents the vector of length groupSizeK in the nth column of x2. The K axis is sliced from the start position of j x groupSizeK. The value range of j is [0, kLoops), where kLoops = ceil(K / groupSizeK). K indicates the length of the K axis. The length of the last slice can be less than groupSizeK. The bias parameter is included only in the mx quantization mode. For the G-B, B-B, and mx quantization modes, the value combinations of [groupSizeM, groupSizeN, groupSizeK] are [1, 128, 128], [128, 128, 128], and [1, 1, 32], respectively.

      [object Object][object Object][object Object]out=(x1@(x2∗x2Scale))∗yScaleout = (x1@(x2 * x2Scale)) * yScale [object Object]
[object Object]

Each operator has calls. First, aclnnQuantMatmulV5GetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnQuantMatmulV5 is called to perform computation.

[object Object]
[object Object]
[object Object]

  • Parameters

    [object Object]

    • Note: Optional inputs refer to optional quantization parameters, and nullptr can be passed.

    • Ascend 950PR/Ascend 950DT:

      • The subscript "1" in the data type column of the preceding table indicates the data type that is not supported by this series.
      • The input parameters x1 and x2 do not support the INT4 and INT32 types.
      • When x2 is in ND format, if x1 is an empty tensor with m = 0 or x2 is an empty tensor with n = 0, the output is an empty tensor. When x2 is in FRACTAL_NZ format, if x1 is an empty tensor with m = 0, the output is an empty tensor.

    • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

      • The superscript "2" in the "Data Type" column of the table above indicates data types that are not supported by the products.

    • Calculation formula: [object Object][object Object]

      groupSize=groupSizeK∣groupSizeN<<16∣groupSizeM<<32groupSize = groupSizeK | groupSizeN << 16 | groupSizeM << 32

  • Returns

    [object Object]: status code. For details, see .

    The first-phase API implements input parameter verification. The following errors may be thrown:

    [object Object]
[object Object]

  • Parameters

    [object Object]

  • Returns

    [object Object]: status code. For details, see .

[object Object]
  • Deterministic computation:
    • The default deterministic implementation of aclnnQuantMatmulV5 is used.
[object Object][object Object]

  • Common constraints: [object Object][object Object]

    • The current version does not support yScale, x1Offset, x2Offset, and yOffset. You need to pass nullptr.
    [object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]undefined
    • Restrictions on x1:
      • The size of the last dimension of x1 cannot exceed 65535. transposeX1 can only be false.
    • Restrictions on x2:
      • The size of the last dimension of x2 cannot exceed 65535. transposeX2 can only be true.
      • The shape of the input is (batch, k1, n1, n0, k0), where batch is optional, k0 = 32, n0 = 16, and k in x1 and k1 in x2 must meet the following relationship: ceil(k/32) = k1.
      • x2 needs to be processed by to obtain the AI processor affinity data layout format from x2 in ND format.
    • Restrictions on x1Scale: The data format is ND, the shape is 1D (t,), and t = m, where m is the same as that of x1.
    • Restrictions on x2Scale: The data format is ND, the shape is 1D (t,), and t = n, where n is the same as that of x2.
    • Restrictions on bias: The data format is ND, and the shape can be 1D (n,) or 3D (batch, 1, n), where n is the same as that of x2.
    [object Object]
[object Object][object Object][object Object]

  • Common constraints: [object Object][object Object]

    • When transposeX1 is false, the shape of x1 is (batch, m, k). When transposeX1 is true, the shape of X1 is (batch, k, m), where batch may not exist.
    • The constraints for transposeX2 are as follows:
      • In ND format, if transposeX2 is false, the shape of X2 is (batch, k, n); if transposeX2 is true, the shape of X2 is (batch, n, k), where batch may not exist. The value of k is the same as that of k in the shape of x1.
      • In NZ format:
        • If transposeX2 is true, the shape of X2 is (batch, k1, n1, n0, k0), where batch may not exist. The value of k0 is 32, and the value of n0 is 16. The value of k in the shape of x1 and the value of k1 in the shape of x2 must meet the following relationship: ceil(k/32) = k1.
        • If transposeX2 is false, the shape of X2 is (batch, n1, k1, k0, n0), where batch may not exist. The value of k0 is 16, and the value of n0 is 32. The value of k in the shape of x1 and the value of k1 in the shape of x2 must meet the following relationship: ceil(k/16) = k1.
        • You can use the aclnnCalculateMatmulWeightSizeV2 and aclnnTransMatmulWeight APIs to convert the input format from ND to NZ.
    • If the original input type of x2Scale does not meet the constraints in the quantization scenario, call the aclnnTransQuantParamV2 API to convert the scale to the INT64 or UINT64 data type.
    • yScale is not supported in the current version. Pass nullptr.
    • The shape of out supports 2 to 6 dimensions, (batch, m, n), where batch may not exist. The data type can be FLOAT16, INT8, BFLOAT16, or INT32.
    • When x1 and x2 are of type INT8, out is of type INT32, and bias is of type INT32 or nullptr, the actual scale is not involved in the computation. The computation formula is as follows:

      • bias INT32

        out=x1@x2+biasout = x1@x2 + bias

      • No bias

        out=x1@x2out = x1@x2
    [object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]undefined

    • The value relationships of x1 shape, x2 shape, x1Scale shape, x2Scale shape, bias shape, and groupSize are as follows:

      [object Object]undefined

    • Note: In the preceding table, gsM, gsK, and gsN indicate groupSizeM, groupSizeK, and groupSizeN, respectively.

    • Restrictions on x1: Currently, k must be 128-pixel aligned and be a multiple of 4 x 128, and transposeX1 must be false.

    • Restrictions on x2: Currently, n must be 256-pixel aligned, k must be 128-pixel aligned and be a multiple of 4 x 128, and transposeX2 must be true.

    [object Object][object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]undefined

    • Restrictions on x1:

      • When the data type is INT4, transposeX1 is false. The dimension is (m, k), and k must be an even number.
      • When the data type is INT32, transposeX1 is false. Eight INT4 data elements are stored in each INT32 data element. The corresponding dimension is (m, ceil(k / 8)). k must be a multiple of 8.
      • When the data type is INT8 and the data type of x2 is INT32, transposeX1 is false. The dimension is (m, k), and k must be an even number.

    • Restrictions on x2:

      • When the data type is INT4:
        • Currently, only the 2D ND format is supported.
        • If transposeX2 is set to true, the shape is (n, k), where k must be an even number.
        • If transposeX2 is set to false, the shape is (k, n), where n must be an even number.
      • When the data type is INT32, each INT32 data entry stores eight INT4 data entries.
        • Currently, only the 2D ND format is supported.
        • When transposeX2 is true, the dimension is (n, ceil(k / 8)). k must be a multiple of 8.
        • When transposeX2 is false, the dimension is (k, ceil(n / 8)). n must be a multiple of 8.
        • The aclnnConvertWeightToINT4Pack API can be used to convert x2 from INT32 (one int32 space stores one int4 data entry in bits 0–3) to INT32 (one int32 space stores eight int4 data entries) or INT4 (one int4 space stores one int4 data entry). For details, see .

    • Restrictions on x1Scale: The data format is ND, the shape is 1D (t,), and t = m, where m is the same as that of x1.

    • Restrictions on x2Scale: The data format is ND, the shape is 1D (t,), and t = 1 or n, where n is the same as that of x2.

    • Restrictions on x2Offset: The data format is ND, the shape is 1D (t,), and t = 1 or n, where n is the same as that of x2.

    • Restrictions on bias:

      • The data layout can be ND. The shape can be one-dimensional (n,) or three-dimensional (batch, 1, n), where n is the same as n of x2.
      • When x1 and x2 are INT32 or INT4, the shape of bias can only be one-dimensional (n,).
      • When the shape of out is two-, four-, five-, or six-dimensional, the shape of bias can only be one-dimensional (n,).

    • Restrictions on yOffset: The shape can be 1D (n). It is an auxiliary result calculated offline during computation. The value must be 8 x2 x2Scale and accumulated in the first dimension.

    [object Object][object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]

    • The value relationships between x1, x2, x1Scale, x2Scale, and groupSize are as follows:

      [object Object]undefined

    • Restrictions on x1:

      • When the data type is INT8, k must be aligned with 256 and be less than 29576. transposeX1 is false.
      • When the data type is INT4, k must be aligned with 1024. transposeX1 is false.

    • Restrictions on x2:

      • When the data type is INT32, k must be aligned with 256. transposeX2 is false.
      • When the data type is INT4, k must be aligned with 1024 and n must be aligned with 256. transposeX2 is true.

    • Restrictions on x2Scale:

      • When the data type is UINT64 or INT64, the TransQuantParamV2 operator supports only one dimension. Therefore, you need to reshape x2Scale to a one-dimensional view (k / groupSize * n), call the aclnn API of the TransQuantParamV2 operator to convert x2Scale to the UINT64 or INT64 data type, and then reshape the output to a two-dimensional view (k / groupSize, n). The groupSize value is 256.
      • When x1 and x2 are of the INT4 type, the shape of x2Scale is (ceil(k / 256), n).
    [object Object]
[object Object][object Object][object Object]

  • Common constraints: [object Object][object Object]

    • Shape of x1 when transposeX1 is false: (batch, m, k). Shape of x1 when transposeX1 is true: (batch, k, m). The first 0 to 4 dimensions represent batch. Dimension 0 indicates that the batch does not exist.

    • Shape of x2 when transposeX2 is false: (batch, k, n). Shape of x2 when transposeX2 is true: (batch, n, k). The first 0 to 4 dimensions represent batch. Dimension 0 indicates that the batch does not exist. k is the same as k in the shape of x1.

    • If the original input type of x2Scale does not meet the combination requirements in the quantization scenario, call the aclnnTransQuantParamV2 API to convert the scale to the INT64 or UINT64 data type.

    • yScale is supported only when x1 is FLOAT8_E4M3FN and x2 is FLOAT4_E2M1. The shape is 2-dimensional (1, n), where n is the same as that of x2.

    • x2Offset is supported only when the data types of x1 and x2 are both INT8 and the data type of out is INT8. For other input types, nullptr needs to be passed. The shape is 1-dimensional (t,). t can be 1 or n, where n is the same as that of x2.

    • yOffset is a reserved parameter and is not supported in the current version. nullptr or an empty tensor needs to be passed.

    • Restrictions on bias:

      • This parameter is optional. nullptr can be passed.
      • When the shape of out is 2, 4, 5, or 6, the shape of bias can be 1-dimensional (n,) or 2-dimensional (1, n).
      • When the shape of out is 3-dimensional, the shape of bias can be 1-dimensional (n,) or 3-dimensional (batch, 1, n).

    • Restrictions on groupSize:

      • It is valid only in the mx, G-B, B-B, and T-CG quantization modes (../common/Quantization Introduction.md).
      • The value of groupSize is valid only when the input of x1Scale and x2Scale is 2-dimensional or higher. In other scenarios, 0 needs to be passed.
      • The input groupSize is internally decomposed into groupSizeM, groupSizeN, and groupSizeK according to the following formulas. If one or more of them are 0, groupSizeM, groupSizeN, and groupSizeK will be reset based on the input shape of x1, x2, x1Scale, and x2Scale for computation. Principle: Assume that groupSizeM is 0, indicating that the quantization group size in the m direction is inferred by the API. The inference formula is groupSizeM = m/scaleM. (Ensure that m can be exactly divided by scaleM.) m is the same as that in the shape of x1, and scaleM is the same as that in the shape of x1Scale.
      groupSize=groupSizeK∣groupSizeN<<16∣groupSizeM<<32groupSize = groupSizeK | groupSizeN << 16 | groupSizeM << 32

    • The output shape supports 2 to 6 dimensions, (batch, m, n). The batch dimension may not exist. The batch dimensions of x1 and x2 can be broadcast. The output batch is the same as the broadcast batch. m is the same as that of x1, and n is the same as that of x2.

    • When x1 and x2 are of type int8, out is of type int32, and bias is of type int32 or nullptr, the actual scale is not involved in the calculation. The calculation formula is as follows:

      • bias INT32

        out=x1@x2+biasout = x1@x2 + bias

      • No bias

        out=x1@x2out = x1@x2
    [object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]
    • In T-T quantization, the shape of x1Scale is (1,) or nullptr, and the shape of x2Scale is (1,).
    • In T-C quantization, the shape of x1Scale is (1,) or nullptr, and the shape of x2Scale is (n,), where n is the same as that of x2.
    • When the data type of x1/x2 is FLOAT8_E4M3FN/FLOAT8_E5M2/HIFLOAT8, static quantization and dynamic quantization are distinguished. In static quantization, the data type of x2Scale is UINT64 or INT64. In dynamic quantization, the data type of x2Scale is FLOAT32. When the data type of x1/x2 is INT8, dynamic T-C or dynamic T-T quantization is not supported.
    • In dynamic T-C quantization, bias is not supported.
    [object Object][object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]
    • In the K-C quantization scenario, the shape of x1Scale is (m,), and the shape of x2Scale is (n,), where m is the same as that of x1, and n is the same as that of x2.
    • In the K-T quantization scenario, the shape of x1Scale is (m,), and the shape of x2Scale is (1,), where m is the same as that of x1.
    [object Object][object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]

    • The value relationships between x1, x2, x1Scale, x2Scale, and groupSize are as follows:

      [object Object]undefined

    • Note: In the preceding table, gsM, gsK, and gsN indicate groupSizeM, groupSizeK, and groupSizeN, respectively.

    • In G-B and B-B quantization scenarios, the transpose attributes of x1 and x1Scale must be the same, and the transpose attributes of x2 and x2Scale must be the same.

    • In G-B quantization scenarios, bias is supported only when the input is of type int8.

    • In B-B quantization scenarios, the input is of type int8 and bias is not supported.

    [object Object][object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]

    • The value relationships between x1 data type, x2 data type, x1, x2, x1Scale, x2Scale, and groupSize are as follows:

      [object Object]undefined

    • In the mx full quantization scenario, when the data type of x2 is FLOAT8_E4M3FN/FLOAT8_E5M2, the transpose attributes of x1 and x1Scale must be the same, and the transpose attributes of x2 and x2Scale must be the same.

    • In the mx full quantization scenario, when the data type of x2 is FLOAT4_E2M1, only transposeX1 = false and transposeX2 = true are supported. In addition, k must be an even number and ceil(k/32) must be an even number.

    • In the mx fake-quantization scenario, when the data type of x2 is FLOAT4_E2M1, transposeX1 is false and transposeX2 is true, and the batch axis is not supported. The data format can be ND or AI processor affinity format. When the data format is ND, k must be a multiple of 64. When the data format is the AI processor affinity format, both k and n must be multiples of 64.

    • In the mx fake-quantization scenario, bias is an optional parameter. The data type must be BFLOAT16, the data format must be ND, and the shape must be 2D (1, n). If this parameter is not required, pass nullptr.

    • In the mx fake-quantization scenario, the value combinations of [groupSizeM, groupSizeN, groupSizeK] can be [0, 0, 32] and [1, 1, 32], and the corresponding groupSize values are 32 and 4295032864, respectively.

    [object Object][object Object][object Object]
    • The input and output support the following data type combinations:

    [object Object][object Object]

    [object Object]

    • The relationship between x1, x2, x1Scale, x2Scale, and groupSize is as follows:

      [object Object]undefined

    • In T-CG quantization mode, the yScale data type supports INT64 and UINT64, the data format supports ND, and the shape supports 2D (1, n). If the original input data type does not meet the data type combination in the restrictions, you need to call the aclnn API of the TransQuantParamV2 operator to convert the data type to UINT64 in advance. When the input data type is INT64, the INT64 data is processed as UINT64 internally.

    • In T-CG quantization mode, bias is a reserved parameter and is not supported in the current version. Therefore, nullptr needs to be passed.

    • In T-CG quantization mode, transposeX1 is set to false. The data format can be ND or the AI processor affinity data layout format. When the data format is ND, k must be a multiple of 64 and transposeX2 must be true. When the data format is the AI processor affinity data format, k and n must be multiples of 64 and transposeX2 must be false.

    • In T-CG quantization mode, the value combination of [groupSizeM, groupSizeN, groupSizeK] can be [0, 0, 32] or [1, 1, 32], and the corresponding groupSize values are 32 and 4295032864, respectively.

    [object Object]
[object Object][object Object]

The following example is for reference only. For details, see .

  • Ascend 950PR/Ascend 950DT: x1 is of type FLOAT8_E4M3FN, x2 is of type FLOAT4_E2M1, x2Scale is of type BFLOAT16, and yScale is of type UINT64.

    [object Object]

  • Ascend 950PR/Ascend 950DT: x1 is of type INT8, x2 is of type INT8, x1Scale is of type FLOAT32, x2Scale is of type FLOAT32, and bias is of type INT32.

    [object Object]

  • Ascend 950PR/Ascend 950DT: x1 and x2 are FLOAT8_E4M3FN, x1Scale and x2Scale are FLOAT32, x2Offset is not available, and bias is FLOAT32.

    [object Object]

  • [object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: x1 is INT8, x2 is INT32, x1Scale is FLOAT32, and x2Scale is UINT64

    [object Object]
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details