Overview

This section describes the usage scenarios of torch module-based PTQ.

This feature is supported only by the following products. Ensure that PyTorch 2.7.1 is used. For details, see Table 1.

Ascend 950PR/Ascend 950DT

Atlas A3 training products/Atlas A3 inference products

Atlas A2 training products/Atlas A2 inference products

Scenario

The differences between torch module-based PTQ and Graph-based Quantization are as follows:

Graph-based compression requires the PyTorch model to be exportable to ONNX. It enables graph fusion operations such as Conv+BN and Matmul+Add, resulting in better network performance after compression. In contrast, torch module-based PTQ is not subject to these constraints.

Prerequisites

All operators in the model to be quantized must be supported by the NPU. If unsupported operators are present, the quantized deployable model will fail to infer on the NPU. To verify operator support, ensure that the PyTorch training script can run successfully on the NPU before quantization. For details, see Traditional Model Performance Tuning Guide.

Parent topic: Torch Module-based Quantization