Documentation

AMCT (ONNX)

This section uses Ubuntu 20.04 as an example to describe the pre-installation actions. The version mapping is as follows:

Table 1 Version mapping

Type

Version Requirement

How to Obtain

Restriction

OS and version

EulerOS release 2.0 (SP10) aarch64

cat /etc/*release && uname -m

Only quantization on CPU is supported.

OS and version

Ubuntu 20.04 x86_64

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m

Quantization on CPU or GPU is supported.

OS and version

Ubuntu 20.04 aarch64

Download the required software release from the Ubuntu official website and install it. Run the following command to query the installation:

cat /etc/*release && uname -m

Quantization on CPU or GPU is supported.

ONNX and opset

ONNX 1.14.0, 1.10.0, 1.9.0, or 1.8.0.

ONNX 1.14.0 is recommended.

  • ONNX 1.14.0 works with opset versions 16, ONNX Runtime 1.16.0, and Python 3.10.0.
  • ONNX 1.10.0 works with opset versions 8–14, ONNX Runtime 1.9.0, and Python 3.9.2.
  • ONNX 1.9.0 works with opset versions 8–14, ONNX Runtime 1.8.0, and Python 3.9.2.
  • ONNX 1.8.0 works with opset versions 8–13, ONNX Runtime 1.6.0, and Python 3.7.5.

For details, see Dependency Installation.
  • Only operators of opset versions 8–14 in the ai.onnx domain are supported.
  • ONNX Runtime 1.8.0 has the following known issue: When it is deployed on the 12th Gen Intel Core processors with the hybrid architecture including Performance-cores (P-cores) and Efficient-cores (E-cores), segmentation errors may occur. For details and other known issues, click here.
  • ONNX Runtime 1.6.0 supports quantization only for FP32 data type.

ONNX Runtime

1.16.0, 1.9.0, 1.8.0, 1.6.0

ONNX Runtime framework. For details, see Dependency Installation.

CUDA and cuDNN
  • ONNX Runtime 1.16.0 works with

    CUDA 11.8 and cuDNN 8.2.4–8.9.0.

  • ONNX Runtime 1.9.0 works with

    CUDA 11.4 and cuDNN 8.2.4.

  • ONNX Runtime 1.8.0 works with

    CUDA 11.0 and cuDNN 8.0.5.

Prepare related software packages and install them.

Link for obtaining CUDA: https://developer.nvidia.com/cuda-toolkit-archive

cuDNN: https://developer.nvidia.com/cudnn

If quantization is performed in ONNX Runtime (GPU), CUDA and cuDNN must be installed.

Python

Python3.7.x, Python3.8.x, Python3.9.x, Python3.10.x, Python3.11.x

Python 3.10.0 is recommended.

For Ubuntu, see Python 3.9.2 Installation on Ubuntu.

For EulerOS, see Python Python3.9.2 Installation on EulerOS.
  • Ensure that the server has Internet access.
  • This document uses Python 3.9.2 as an example. The environment variables and installation commands are subject to the actual Python version.

numpy
  • ONNX 1.14.0 works with NumPy 1.21.6–1.26.4.
  • ONNX 1.10.0, 1.9.0, and 1.8.0 works with NumPy 1.20.0–1.23.5.

For details, see Dependency Installation.

protobuf
  • ONNX 1.14.0 works with Protobuf 3.20.2–3.20.3.
  • ONNX 1.10.0, 1.9.0, and 1.8.0 works with Protobuf 3.13.0+.