Quantifiable Layers and Restrictions

This section describes the quantifiable layers of different frameworks and related restrictions.

If the input data type or weight data type of the network model is float16 or mixed precision of float32 and float16, the quantization of the following operators is disabled:
AvgPool, Pooling, AvgPoolV2, MaxPool, MaxPoolV3, Add, Eltwise, and BatchMatMulV2 (Both inputs are tensors.).

The following products do not support the Caffe framework:
Atlas A2 training product/Atlas A2 inference product

Atlas A3 training product/Atlas A3 inference product

Atlas 350 Accelerator Card

Layers that support uniform quantization and their restrictions (Caffe framework)

Supported Layer Type	Restrictions	Ascend IR–defined Layer
InnerProduct	transpose = false, axis = 1	FullyConnection
Convolution	4 × 4 filter	Conv2D
Deconvolution	1-dilated 4 × 4 filter	Deconvolution
Pooling	If mode is set to 1, indicating full quantization (weight+tensor), and global_pooling is set to false, the N shift operation is not supported. If mode is set to 0, only tensor quantization is performed.	Pooling
Eltwise	Only tensor quantization is performed and operation=1 is required.	Eltwise

Layers that support uniform quantization and their restrictions (TensorFlow framework)

Supported Layer Type	Restrictions	Ascend IR–defined Layer
MatMul	transpose_a, transpose_b, adjoint_a, and adjoint_b must be set to False. The weights do not have dynamic inputs (such as placeholders).	MatMulV2
Conv2D	The weights do not have dynamic inputs (such as placeholders).	Conv2D
DepthwiseConv2dNative	The weights do not have dynamic inputs (such as placeholders).	DepthwiseConv2D
Conv2DBackpropInput	If dilation is set to 1, the weights do not have dynamic inputs (such as placeholders).	Conv2DBackpropInput
BatchMatMulV2	adj_x=False When the second input is a constant, only two dimensions are supported. When both inputs are tensors, only INT8 symmetric quantization is supported. If both inputs are tensors, benefits can be obtained only in the following products. For other products, the precision reduces after quantization. Atlas 200I/500 A2 inference product Atlas A2 training product/Atlas A2 inference product Atlas A3 training product/Atlas A3 inference product Atlas 350 Accelerator Card	BatchMatMulV2
AvgPool	The N shift operation is not supported.	AvgPool
Conv3D	dilation_d=1	Conv3D
MaxPool	Tensor quantization only	MaxPool, MaxPoolV3
Add	Only tensor quantization is performed, and only single-input quantization is supported.	Add

Layers that support uniform quantization and their restrictions (ONNX framework)

Supported Layer Type	Restrictions	Ascend IR–defined Layer
Conv	1-dilated 5 × 5 filter The weights do not have dynamic inputs (such as placeholders).	Conv2D, Conv3D
Gemm	transpose_a=false The weights do not have dynamic inputs (such as placeholders).	MatMulV2
ConvTranspose	1-dilated 4 × 4 filter The weights do not have dynamic inputs (such as placeholders).	Conv2DTranspose
MatMul	When the second input is a constant, only two dimensions are supported. When both inputs are tensors, only INT8 symmetric quantization is supported. In the quantization scenario where both inputs are tensors, benefits can be obtained only in the following products. For other products, the precision reduces after quantization. Atlas 200I/500 A2 inference product Atlas A2 training product/Atlas A2 inference product Atlas A3 training product/Atlas A3 inference product Atlas 350 Accelerator Card The weights do not have dynamic inputs (such as placeholders).	BatchMatMulV2
AveragePool	If global_pooling is set to false, the N shift operation is not supported.	AvgPoolV2
MaxPool	Tensor quantization only	MaxPool, MaxPoolV3
Add	Only tensor quantization is performed, and only single-input quantization is supported.	Add

This version does not support weight-only quantization.

Weight-only quantization is supported only by the following product types:

Atlas inference product

Atlas A2 training product/Atlas A2 inference product

**Table 1** Layers that support weight-only quantization and restrictions
Ascend IR–defined Layer	Weight Quantization Only, channel_wise=true in Weight ARQ	Weight Quantization Only, asymmetric in Weight ARQ (true/false)	Weight and Activation Quantization, channel_wise=true in Weight ARQ	Weight and Activation Quantization, asymmetric=true in Weight ARQ	Restriction
MatMulV2	√	true	×	×	The second inputs do not have dynamic inputs (such as placeholders).
BatchMatMulV2	√	true	×	×	The second inputs do not have dynamic inputs (such as placeholders).
FFN	√	true and false	×	×	The input expert_tokens of the FFN operator is not empty. The two weights of the FFN operator are constants of Float16. The antiquant_scale1, antiquant_scale2, antiquant_offset1, and antiquant_offset2 inputs of the FFN operator are empty. The weight cannot be shared. FFN operator quantization is supported only by the Atlas A2 training product/Atlas A2 inference product.

Notes:

√: Supported. ×: Quantization is abnormal.
channel_wise=true in Weight ARQ: Channels are separately quantized using different quantization factors.
asymmetric in Weight ARQ
- true: Asymmetric weight quantization is used.
- false: Symmetric weight quantization is used.
- true and false: Both symmetric weight quantization and asymmetric weight quantization are supported.

Parent topic: Quantization Configuration