Workflow

Obtain the package.

Obtain the tool package. For details, see Package Preparation.

Prepare for installation.

Before installing AMCT, create an AMCT installation user, check the OS environment, install dependencies, and upload the package. For details, see Installation Preparations.

The environment configuration varies depending on the framework. For details, see the corresponding framework.

Install.

Install AMCT by referring to Tool Installation. The installation command varies depending on the framework. For details, see the installation procedure of the corresponding framework.

Perform post-installation actions.

After installing AMCT, perform related operations by referring to Post-installation Actions. If the corresponding frameworks described in this section are not available, skip this step.

(Optional) Create a script to use AMCT APIs.

If the sample provided by AMCT is used for model compression, the APIs in this document can be directly called.

If you need to compress your own network model and do not use the sample link provided in this document for compression, you need to modify the compression script for adaptation before compression.

Compress.

Perform the compression operation.

For the quantization feature, AMCT provides two quantization methods: CLI-based quantization and Python API–based quantization. For details about their differences, see Table 1.

• CLI-based quantization: You only need to prepare the model and its datasets. Currently, only some functions are supported.
• Python API-based quantization: You need to understand the Python syntax and detailed quantization process. All quantization modes are supported.

Run the provided quantization script or CLI to quantize your original network model with the prepared datasets.

AMCT is developed based on deep learning frameworks. During compression, call the deep learning framework in use for inference or training.

(Follow-up) Run inference on the compressed model.

You can use ATC to convert the compressed deployable model into an offline model adapted to the AI processor, and then perform subsequent inference.

Set up the online inference environment powered by NPUs.

Perform the following steps to set up an online inference environment powered by NPUs.

1. Install the driver, firmware, and CANN package by referring to CANN Software Installation.
2. Install the TF Adapter plugin package by referring to "Environment Setup > Installing the Framework Plugin Package" in TensorFlow 1.15 Model Porting Guide or TensorFlow 2.6.5 Model Porting Guide.

Install the CPU version of TensorFlow.

The online inference environment supports only quantization on the NPU as opposed to the GPU. As such, you only need to install the CPU version of TensorFlow. For details, see AMCT (TensorFlow, Ascend).

Install AMCT.

Install TensorFlow AMCT by referring to Tool Installation. Before installation, obtain the package, create an AMCT installation user, check the environment, install dependencies, and upload the package.

(Optional) Create a script to use AMCT APIs.

If the sample provided by AMCT is used for model compression, the APIs in this document can be directly called.

If you need to compress your own network model and do not use the sample link provided in this document for compression, you need to modify the compression script for adaptation before compression.

Quantize.

Run the provided quantization script to quantize your original network model with the prepared datasets.

AMCT is developed based on deep learning frameworks. During quantization, call the deep learning framework in use for inference.

(Follow-up) Run inference on the quantized model.

The quantized .pb model can serve for online inference in the NPU environment. For details about how to perform online inference, see TensorFlow 1.15 Model Porting Guide or TensorFlow 2.6.5 Model Porting Guide based on your TensorFlow version.