Documentation

AMCT (PyTorch)

This section uses Ubuntu 20.04 as an example to describe the pre-installation actions. The version mapping is as follows:

Table 1 Version mapping

Type

Version Requirement

How to Obtain

Restriction

OS and version

EulerOS release 2.0 (SP10) aarch64

cat /etc/*release && uname -m
  • Only quantization on CPU is supported.
  • If the PyTorch-based AMCT package is installed, eight threads are used for compilation by default. Therefore, the OS running memory must be greater than or equal to 12 GB.

OS and version

Ubuntu 20.04 x86_64

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m
  • Quantization on CPU or GPU is supported.
  • If the PyTorch-based AMCT package is installed, eight threads are used for compilation by default. Therefore, the OS running memory must be greater than or equal to 12 GB.

OS and version

Ubuntu 20.04 aarch64

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m
  • Quantization on CPU or GPU is supported.
  • If the PyTorch-based AMCT package is installed, eight threads are used for compilation by default. Therefore, the OS running memory must be greater than or equal to 12 GB.

PyTorch

2.1.0, 1.10.0, 1.8.0, 1.5.0, 1.4.0

  • PyTorch 2.1.0 works with ONNX 1.14.0, ONNX Runtime 1.16.0 and Python 3.10.0.
  • PyTorch 1.10.0 works with ONNX 1.9.0, ONNX Runtime 1.8.0, and Python 3.9.2.
  • PyTorch 1.8.0, 1.5.0, and 1.4.0 work with ONNX 1.8.0, ONNX Runtime 1.6.0, and Python 3.7.5.

Install the CPU or GPU version as needed. For details, see Dependency Installation.
  • Version 1.4.0 is not recommended due to disturbing warnings.
  • PyTorch 1.8.0 and earlier versions do not support the compression of models whose size is greater than or equal to 2GB.

CUDA toolkit/CUDA driver

11.8, 11.1, 10.2

  • PyTorch 2.1.0 works with CUDA 11.8.
  • PyTorch 1.10.0 works with CUDA 11.1.
  • PyTorch 1.8.0, 1.5.0, and 1.4.0 work with CUDA 10.2.

Obtain required packages for installation. For example, you can obtain the Toolkit package from the following URL, which contains the Driver package.

https://developer.nvidia.com/cuda-toolkit-archive

You need to install the CUDA software to perform quantization on the GPU.

ONNX

1.14.0, 1.9.0, 1.8.0

For details, see Dependency Installation.
  • Ensure that the server has Internet access.
  • This document uses Python 3.9.2 as an example. The environment variables and installation commands are subject to the actual Python version.
  • For known issues of ONNX Runtime 1.8.0, click here.

ONNX Runtime

1.16.0, 1.8.0, 1.6.0

Python

Python3.7.x, Python3.8.x, Python3.9.x, Python3.10.x, Python3.11.x

3.10.0 is recommended.

For Ubuntu, see Python 3.9.2 Installation on Ubuntu.

For EulerOS, see Python Python3.9.2 Installation on EulerOS.

numpy
  • PyTorch 2.1.0 works with NumPy 1.21.6 and later versions.
  • PyTorch 1.10.0, 1.8.0, 1.5.0, and 1.4.0 work with NumPy 1.20.0–1.23.5.

For details, see Dependency Installation.

protobuf
  • PyTorch 2.1.0 works with Protobuf 3.20.2 and later versions.
  • PyTorch 1.10.0, 1.8.0, 1.5.0, and 1.4.0 work with Protobuf 3.13.0 and later versions.