Note: When using this API, ensure that the driver firmware package and CANN package are in the 8.0.RC2 version or later. Otherwise, an error, such as BUS ERROR, will be reported.
API function: performs fake-quantization on the input x2 and then performs Matmul and AllReduce computation. The perTensor, perChannel, and perGroup quantization modes are supported.
Formula:
Each operator has [two-phase API](../common/two_phase_api .md) calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.
Parameters
[object Object]Returns:
[object Object]: status code. For details, see .The first-phase API implements input parameter verification. The following errors may be thrown.
[object Object]
- Deterministic computation:
- [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: The aclnnWeightQuantMatmulAllReduce function is implemented in non-deterministic mode by default. You can enable deterministic computing by setting the HCCL_DETERMINISTIC environment variable to true.
- Ascend 950PR/Ascend 950DT: The aclnnWeightQuantMatmulAllReduce function is implemented in deterministic mode by default.
- MC2 is disabled in incremental scenarios but enabled in full scenarios.
- The input
[object Object]can be two-dimensional or three-dimensional, with shape (b, s, k) or (m, k), respectively. [object Object]must be 2D The shape is (k, n). The k axis meets the input parameter requirements of the MM operator, and the k axis is equal. The value range of m is [1, 2147483647], and the value range of k and n is [1, 65535].- The passed x1, x2, antiquantScale, or output cannot be a null pointer.
- When the shape of the input
[object Object]is (b, s, k), the shape of[object Object](non-null scenario) and the shape of[object Object]are (b, s, n). When the shape of the input[object Object]is (m, k), the shape of[object Object](non-null scenario) and the shape of[object Object]are (m, n). - If bias is not empty, the shape size is the same as the last dimension of output. In the pertensor scenario, the shape of antiquantScale is (1). In the perchannel scenario, the shape of antiquantScale is (1, n)/(n). In the pergroup scenario, the shape of antiquantScale is (ceil(k,antiquantGroupSize), n). If antiquantOffset is not empty, its shape is the same as that of antiquantScale.
- The data types and data formats of x1, x2, x3 (non-empty), antiquantScale, antiquantOffset (non-empty), output, and bias (non-empty) must be supported.
- The data types of x1, antiquantScale, antiquantOffset (non-null scenario), x3 (non-null scenario), bias (non-null scenario), and output are the same. The value of antiquantGroupSize must be a multiple of 32 and within the value range.
- In the pergroup scenario, when x2 is transposed, antiquantScale and antiquantOffset must be transposed together to ensure continuity.
- In the long sequence scenario, as b/s or m increases, out of memory or computation timeout may occur.
- Only the all-mesh networking of HCCS links is supported.
- [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]: 1, 2, 4, and 8 ranks are supported.
- Ascend 950PR/Ascend 950DT: 1, 2, 4, 8, 16, 32, or 64 cards are supported.
- [object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
- The merged compute and communication (MC2) operators in a model support only the same communicator.
- The data format of x2 can be ND (only 2D input is supported in the current version) or FRACTAL_NZ (only 4D input is supported in the current version). When the data format of x2 is FRACTAL_NZ, aclnnCalculateMatmulWeightSizeV2 and aclnnTransMatmulWeight are used together to convert the input from ND to NZ. Only the transpose scenario is supported for non-contiguous tensors.
- Ascend 950PR/Ascend 950DT:
- The data format of x2 can be ND (only 2D input is supported). In the current version, if the data type is INT8, N and K must be 32-byte aligned. If the data type is INT4, N and K must be 64-byte aligned.
- Support for empty tensors:
- Only the scenario where k is 0 is supported. The output is bias + x3. Empty tensor input with bs/m/n being 0 is not supported.
The following table describes the supported input and output data type combinations.
[object Object]Atlas A2 training products/Atlas A2 inference products[object Object]:
[object Object]Ascend 950PR/Ascend 950DT
[object Object]
The following example is for reference only. For details, see .
Note: This sample code calls some HCCL collective communication library APIs, including HcclGetCommName, HcclCommInitAll, and HcclCommDestroy. For details, see .
[object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and Ascend 950 PR/Ascend 950 DT:
[object Object]