Preprocesses the input weight data to convert the layout of low-bit data from sparse storage to dense storage. When the of the output weightInt4Pack is set to FRACTAL_NZ, this operator converts the from ND to FRACTAL_NZ.
[object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The input weight data of type INT32 is packed into INT4 data in a compact layout.
Ascend 950PR/Ascend 950DT: packs the weight data of the INT32 type into tightly packed INT4 data and packs the weight data of the FLOAT type into tightly packed FLOAT4_E2M1 data.
Each operator has calls. First, aclnnConvertWeightToINT4PackGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation flow. Then, aclnnConvertWeightToINT4Pack is called to perform computation.
Parameters
[object Object]Returns
[object Object]: status code. For details, see .The first-phase API implements input parameter verification. The following errors may be thrown.
[object Object]
Deterministic description:
- [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], and Ascend 950PR/Ascend 950DT: The default deterministic implementation of aclnnConvertWeightToINT4Pack is used.
The relationships between data types and data formats of the parameters are listed as follows.
[object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:
[object Object]Ascend 950PR/Ascend 950DT:
[object Object]
[object Object]Atlas A2 training products/Atlas A2 inference products[object Object] and [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]: The following example is for reference only. For details, see . aclnnWeightQuantBatchMatmulV2 and aclnnWeightQuantBatchMatmulV3 are available for fake quantization. aclnnWeightQuantBatchMatmulV2 is used as an example.
[object Object]Ascend 950PR/Ascend 950DT:
The following example (INT32 input) is for reference only. For details, see . The fake-quantization APIs include aclnnWeightQuantBatchMatmulV2 and aclnnWeightQuantBatchMatmulV3. aclnnWeightQuantBatchMatmulV2 is used as an example here.