Quantization and Compression

COMPRESSION_OPTIMIZE_CONF

Sets the directory (including the file name) of the compression configuration file. This parameter is used to enable the compression optimization feature specified in the configuration file to improve network performance, for example, /home/test/compression_optimize.cfg. The corresponding options parameter is ge.compressionOptimizeConf.

Configuration example:

{ge::ir_option::COMPRESSION_OPTIMIZE_CONF, "/home/test/compression_optimize.cfg"}

An example of the file contents is as follows:

enable_first_layer_quantization:true

• This file supports configuration of only the enable_first_layer_quantization feature, which specifies whether to optimize the convolution at the first layer of the AIPP. (The AIPP is merged with the Quant operator before the CONV2D convolution is performed at the first layer of the model obtained after quantization.)

  After the enable_first_layer_quantization feature is enabled, performance is improved only when the AIPP+CONV2D structure exists in the network structure and ENABLE_SMALL_CHANNEL is set to 1 during model compilation. The precision of the quantized model is compromised to some extent. You can determine whether to enable this feature as required.

• In the configuration file, the name of a compression feature is followed by a value, either true (feature enabled) or false (feature disabled; default). The feature name and the value are separated with a colon (:).

Applicable products:

• Ascend 950PR/Ascend 950DT: not supported
• Atlas A3 training products/Atlas A3 inference products: not supported
• Atlas A2 training products/Atlas A2 inference products: not supported
• Atlas 200I/500 A2 inference products: supported
• Atlas inference products: supported
• Atlas training products: not supported

SPARSITY

Enables global sparsity. The corresponding options parameter is ge.enableSparseMatrixWeight.

In the model output by AMCT (Ascend Model Compression Toolkit) after 2:4 structured sparsity, there may be the cases that at least two weight elements in the Cin dimension out of four contiguous ones are forced to zero. You can enable global sparsity during model conversion to filter out two elements to reduce computational demand for inference and optimize inference performance.

Due to hardware restrictions, this parameter cannot be used together with ENABLE_COMPRESS_WEIGHT or COMPRESS_WEIGHT_CONF.

Parameter values:

• 1: Enables 2:4 structured sparsity.
• 0 (default): Disables the sparsity feature.

Configuration example:

{ge::ir_option::SPARSITY, "1"}

Restrictions: When using this parameter, ensure that a sparse model is used. You are advised to use the compression combination function of AMCT (TensorFlow) or AMCT (PyTorch). The compression combination requires 2:4 structured sparsity and quantization aware training.

Applicable products:

• Ascend 950PR/Ascend 950DT: not supported
• Atlas A3 training products/Atlas A3 inference products: supported
• Atlas A2 training products/Atlas A2 inference products: supported
• Atlas 200I/500 A2 inference products: supported
• Atlas inference products: not supported
• Atlas training products: not supported