Documentation

Functions

This section describes how to automatically quantize a graph by calling APIs.

Overview

You can call aclgrphCalibration to automatically quantize a non-quantized graph. During quantization, operator fusion is performed on some structures in the model. After the fusion, the graph structure is optimized, thereby improving the network inference performance. For details about the supported fusion functions, see Fusion Support.

Restrictions

Refer to Layers That Support Quantization and Restrictions.

Example

This section uses the network quantization model of the Caffe framework as an example.

  1. Add the header files.
    1
2
3
    		 
    #include "amct/acl_graph_calibration.h"
#include "amct/acl_calibration_configs.h"
#include "amct/amct_error_code.h"
  2. Create a graph object manually or parse an existing graph.
    1
2
3
4
5
6
    		 
    #include "parser/caffe_parser.h"
 
ge::Graph graph("origin");
std::map<ge::AscendString, ge::AscendString> parseOptions;
auto ret = ge::aclgrphParseCaffe("./path/to/resnet50.prototxt",
    "./path/to/resnet50.caffemodel", parseOptions, graph);
  3. Define the parameters required by the aclgrphCalibration API.
    1
2
3
4
    		 
    std::map<ge::AscendString, ge::AscendString> quantizeConfigs = {};
quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::INPUT_DATA_DIR)] = ge::AscendString("./path/to/cali_data.bin");
quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::INPUT_SHAPE)] = ge::AscendString("input:16,224,224,3");
quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::SOC_VERSION)] = ge::AscendString("SOC_VERSION"); // Set SOC_VERSION to the actual Ascend AI Processor model.

    If you need to control the quantization process, you can use the CONFIG_FILE parameter to specify a simplified configuration file for controlling the quantization process. For details about the example of the configuration file, see Simplified Configuration File.

    The following is a configuration example of the CONFIG_FILE parameter:

    1
    		 
    quantizeConfigs[ge::AscendString(amct::aclCaliConfigs::CONFIG_FILE)]=ge::AscendString("./calibration.cfg");
  4. Quantize the graph.
    1
2
3
4
5
    		 
    ret = ge::aclgrphCalibration(graph, quantizeConfigs);
if (ret != ge::GRAPH_SUCCESS) {
    return FAILED;
}
return SUCCESS;

When compiling the preceding script, you need to link libamctacl.so and add the search path ${ASCEND_PATH}/compiler/lib64 of the dynamic library during linking. ASCEND_PATH is the same as the /ascend-toolkit/latest/ path under the Ascend-CANN-toolkit installation directory.