AMCT (ONNX)

OS and version

Ubuntu 20.04 (x86_64)

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m

Quantization on the CPU or GPU is supported.

OS and version

Ubuntu 20.04 (AArch64)

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m

Quantization on the CPU or GPU is supported.

ONNX and opset

1.18.0, 1.14.0, 1.10.0, 1.9.0, 1.8.0

1.18.0 is recommended.

• ONNX 1.18.0 works with Opset v18, ONNX Runtime 1.20.0, and Python 3.12/3.13.
• ONNX 1.14.0 works with Opset v16, ONNX Runtime 1.16.0, and Python 3.10.0/Python 3.11.x.
• ONNX 1.10.0 works with opset versions 8–14, ONNX Runtime 1.9.0, and Python 3.9.2.
• ONNX 1.9.0 works with opset versions 8–14, ONNX Runtime 1.8.0, and Python 3.9.2.
• ONNX 1.8.0 works with opset versions 8–13, ONNX Runtime 1.6.0, and Python 3.7.5.

For details, see Installing Dependencies.

• Only operators of opset versions 8–14 in the ai.onnx domain are supported.
• ONNX Runtime 1.8.0 has the following known issue: When it is deployed on the 12th Gen Intel Core processors with the hybrid architecture including Performance-cores (P-cores) and Efficient-cores (E-cores), segmentation errors may occur. For details and other known issues, click here.
• ONNX Runtime 1.6.0 supports quantization only for float32 data type.

ONNX Runtime

1.20.0, 1.16.0, 1.9.0, 1.8.0, 1.6.0

ONNX Runtime framework. For details, see Installing Dependencies.

CUDA and cuDNN

• ONNX Runtime 1.20.0 works with

  CUDA 12.6 and cuDNN 8.9.0–8.9.7.

• ONNX Runtime 1.16.0 works with

  CUDA 11.8 and cuDNN 8.2.4–8.9.0.

• ONNX Runtime 1.9.0 works with

  CUDA 11.4 and cuDNN 8.2.4.

• ONNX Runtime 1.8.0 works with

  CUDA 11.0 and cuDNN 8.0.5.

Prepare related packages and install them.

Link for obtaining CUDA: https://developer.nvidia.com/cuda-toolkit-archive

Link for obtaining cuDNN: https://developer.nvidia.com/zh-cn/cudnn

• If quantization is performed in ONNX Runtime (GPU), CUDA and cuDNN must be installed.

Python

Python 3.7.x, Python 3.8.x, Python 3.9.x, Python 3.10.x, Python 3.11.x, Python 3.12/3.13

For Ubuntu OS, see Python 3.9.2 Installation on Ubuntu.

For EulerOS, see Python3.9.2 Installation on EulerOS.

• Ensure that the server has Internet access when installing dependencies.
• This document uses Python 3.9.2 as an example. The environment variables and installation commands are subject to the actual Python version.

NumPy

• ONNX 1.18.0 works with NumPy 2.0.0 and later versions.
• ONNX 1.14.0 works with NumPy 1.21.6–1.26.4.
• ONNX 1.10.0, 1.9.0, and 1.8.0 work with NumPy 1.20.0–1.23.5.

For details, see Installing Dependencies.

Protobuf

• ONNX 1.18.0 works with Protobuf 3.20.3.
• ONNX 1.14.0 works with Protobuf 3.20.2–3.20.3.
• ONNX 1.10.0, 1.9.0, and 1.8.0 work with Protobuf 3.20.2 and later versions.