Functions

This section describes how to automatically quantize a graph by calling APIs.

Overview

You can call aclgrphCalibration to automatically quantize a non-quantized graph. During quantization, operator fusion is performed on some structures in the model. After the fusion, the graph structure is optimized, thereby improving the network inference performance. For details about the supported fusion functions, see Fusion Support.

Restrictions

Refer to Layers That Support Quantization and Their Restrictions.

Examples

This section uses the network quantization model of the TensorFlow framework as an example.

Add the header files.

        
             #include "amct/acl_graph_calibration.h"
#include "amct/acl_calibration_configs.h"
#include "amct/amct_error_code.h"

Create a graph object manually or parse an existing graph.

        
             #include "parser/tensorflow_parser.h"
 
ge::Graph graph("origin");
std::map<ge::AscendString, ge::AscendString> parseOptions;
auto ret = ge::aclgrphParseTensorFlow("./path/tf_test.pb", parseOptions, graph);

Define the parameters required by the aclgrphCalibration API.

        
             std::map<ge::AscendString, ge::AscendString> quantizeConfigs = {};
quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::INPUT_DATA_DIR)] = ge::AscendString("./path/cali_data.bin");
quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::INPUT_SHAPE)] = ge::AscendString("input:16,224,224,3");
quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::SOC_VERSION)] = ge::AscendString("SOC_VERSION"); // Set SOC_VERSION to the actual Ascend AI Processor model.

If you need to control the quantization process, you can use the CONFIG_FILE parameter to specify a simplified configuration file for controlling the quantization process. For details about the example of the configuration file, see Simplified Quantization Configuration File.

The following is a configuration example of the CONFIG_FILE parameter:

        
             quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::CONFIG_FILE)]=ge::AscendString("./calibration.cfg");

Quantize the graph.

        
             ret = ge::aclgrphCalibration(graph, quantizeConfigs);
if (ret != ge::GRAPH_SUCCESS) {
    return FAILED;
}
return SUCCESS;

When compiling the script of the preceding program, you need to link libamctacl.so and add the search path ${ASCEND_PATH}/compiler/lib64 of the dynamic library. ASCEND_PATH indicates the installation path of the CANN software package. The default installation path /usr/local/Ascend/cann of user root is used as an example.

Parent topic: Automatic Quantization