AMCT (PyTorch)

OS and version

EulerOS release 2.0 (SP10) (AArch64)

cat /etc/*release && uname -m

• Only quantization on the CPU is supported.
• If the PyTorch-based AMCT package is installed, eight threads are used for building by default. Therefore, the OS running memory must be greater than or equal to 12 GB.

OS and version

Ubuntu 20.04 (x86_64)

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m

• Quantization on the CPU or GPU is supported.
• If the PyTorch-based AMCT package is installed, eight threads are used for building by default. Therefore, the OS running memory must be greater than or equal to 12 GB.

OS and version

Ubuntu 20.04 (AArch64)

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m

• Quantization on the CPU or GPU is supported.
• If the PyTorch-based AMCT package is installed, eight threads are used for building by default. Therefore, the OS running memory must be greater than or equal to 12 GB.

PyTorch

2.7.1, 2.1.0, 1.10.0, 1.8.0, 1.5.0, 1.4.0

• PyTorch 2.7.1 works with ONNX 1.18.0, ONNX Runtime 1.20.0, and Python 3.12/3.13.
• PyTorch 2.7.1 works with ONNX 1.16.1, ONNX Runtime 1.20.0, and Python 3.11.x.
• PyTorch 2.1.0 works with ONNX 1.14.0, ONNX Runtime 1.16.0, and Python 3.10.0 or Python3.11.x.
• PyTorch 1.10.0 works with ONNX 1.9.0, ONNX Runtime 1.8.0, and Python 3.9.2.
• PyTorch 1.8.0, 1.5.0, and 1.4.0 work with ONNX 1.8.0, ONNX Runtime 1.6.0, and Python 3.7.5.

Install the CPU or GPU version as needed. For details, see Installing Dependencies.

• Version 1.4.0 is not recommended due to disturbing warnings.
• PyTorch 1.8.0 and earlier versions do not support compression of models whose size is greater than or equal to 2 GB.

NPU environment

Prepare a hardware environment with the Ascend AI Processor and install the driver, firmware, and CANN package. For details, see CANN Software Installation Guide.

You need to install the driver, firmware, and CANN package to perform quantization on the NPU.

This scenario is supported only by the Atlas A2 training products/Atlas A2 inference products.

CUDA Toolkit/CUDA Driver

12.6, 11.8, 11.1, 10.2

• PyTorch 2.7.1 works with CUDA 12.6.
• PyTorch 2.1.0 works with CUDA 11.8.
• PyTorch 1.10.0 works with CUDA 11.1.
• PyTorch 1.8.0, 1.5.0, and 1.4.0 work with CUDA 10.2.

Obtain required packages for installation. For example, you can obtain the Toolkit package from the following URL, which contains the Driver package.

https://developer.nvidia.com/cuda-toolkit-archive

• You need to install the CUDA software to perform quantization on the GPU.

ONNX

1.18.0, 1.16.1, 1.14.0, 1.9.0, 1.8.0

For details, see Installing Dependencies.

• Ensure that the server has Internet access when installing dependencies.
• This document uses Python 3.9.2 as an example. The environment variables and installation commands are subject to the actual Python version.
• For known issues of ONNX Runtime 1.8.0, click here.

ONNX Runtime

1.20.0, 1.16.0, 1.8.0, 1.6.0

Python

Python 3.7.x, Python 3.8.x, Python 3.9.x, Python 3.10.x, Python 3.11.x, Python 3.12/3.13

For Ubuntu OS, see Python 3.9.2 Installation on Ubuntu.

For EulerOS, see Python3.9.2 Installation on EulerOS.

NumPy

• PyTorch 2.7.1 and Python 3.12/3.13 work with NumPy 2.0.0 and later versions.
• PyTorch 2.7.1 and Python 3.11.x work with NumPy 1.24.2–1.26.4.
• PyTorch 2.1.0 works with NumPy 1.21.6 and later versions.
• PyTorch 1.10.0, 1.8.0, 1.5.0, and 1.4.0 work with NumPy 1.20.0–1.23.5.

For details, see Installing Dependencies.

Protobuf

• PyTorch 2.7.1 and Python 3.12/3.13 work with Protobuf 3.20.3.
• PyTorch 2.7.1 and Python 3.11.x work with Protobuf 3.20.2–3.20.3.
• PyTorch 2.1.0, 1.10.0, 1.8.0, 1.5.0, and 1.4.0 work with Protobuf 3.20.2 and later versions.