Installing CANN Packages

Log in to the server as the installation user, upload the CANN software package to be installed to any path (for example, /home/package) in the installation environment, and perform the installation.

The procedure for installing the CANN software package as the root user is the same as that as a non-root user. The following uses a non-root user as an example.

Installing the Toolkit Development Kit

CANN Toolkit development kit, which is installed in the training, inference, and development and debugging scenarios. It is used for training and inference services, model conversion, and operator/application/model development and build.

Ensure that the available space of the installation directory is greater than 7 GB. If the available space is less than 9 GB, clear the space or change the installation directory.

Add the execute permission on the software package and verify the consistency and integrity of the software package installation file.
1 2
chmod +x Ascend-cann-toolkit_<version>_linux-<arch>.run ./Ascend-cann-toolkit_<version>_linux-<arch>.run --check
If the following information is displayed, the software package verification is successful:
1
Verifying archive integrity... 100% SHA256 checksums are OK. All good.
Install the software package. (The installation command supports parameters such as --install-path= <path>. For details, see the parameter description.)
1
./Ascend-cann-toolkit_<version>_linux-<arch>.run --install
After the preceding command is executed, enter Y or y to agree to the Huawei Enterprise End User License Agreement (EULA) to continue the installation.
If the following information is displayed, the software is successfully installed:
1
xxx install success
xxx indicates the name of a software package to be installed.

If you do not specify an installation path, the software is installed in the default path. The default installation paths are as follows: For the root user, the path is /usr/local/Ascend. For a non-root user, the path is ${HOME} /Ascend. ${HOME} indicates the directory of the current user.
Configure environment variables. The default installation path of a non-root user is used as an example. Run the following command based on the actual path of set_env.sh:
1
source ${HOME}/Ascend/ascend-toolkit/set_env.sh
Verifying the installation. Run the following command to query the CANN version. If the query result is the same as the version of the installation software package, the installation is successful.
1. Go to the directory where the software package installation information file is stored. Replace it with the actual installation path. {arch} indicates the CPU architecture (aarch64 or x86_64).
  1
  cd ${HOME}/Ascend/ascend-toolkit/latest/<arch>-linux
2. Run the following command to view the version information provided in the version field:
  1
  cat ascend_toolkit_install.info

Installing the NNAE Deep Learning Engine Package

CANN NNAE deep learning engine package, which is installed in the training and inference scenarios and is used for training and inference services.

Add the execute permission on the software package and verify the consistency and integrity of the software package installation file.
1 2
chmod +x Ascend-cann-nnae_<version>_linux-<arch>.run ./Ascend-cann-nnae_<version>_linux-<arch>.run --check
If the following information is displayed, the software package verification is successful:
1
Verifying archive integrity... 100% SHA256 checksums are OK. All good.
Installs a software package. (The installation command supports parameters such as --install-path= and <path>. For details, see the parameter description.)
1
./Ascend-cann-nnae_<version>_linux-<arch>.run --install
After the preceding command is executed, enter Y or y to agree to the Huawei Enterprise End User License Agreement (EULA) to continue the installation.
If the following information is displayed, the software is successfully installed:
1
xxx install success
xxx indicates the name of a software package to be installed.

If you do not specify an installation path, the software is installed in the default path. The default installation paths are as follows: For the root user, the path is /usr/local/Ascend. For a non-root user, the path is ${HOME} /Ascend. ${HOME} indicates the directory of the current user.
Configure environment variables. The default installation path of a non-root user is used as an example. Run the following command based on the actual path of set_env.sh:
1
source ${HOME}/Ascend/nnae/set_env.sh
Verifying the installation. Run the following command to query the CANN version. If the query result is the same as the version of the installation software package, the installation is successful.
1. Go to the directory where the software package installation information file is stored. Replace it with the actual installation path.
  1
  cd ${HOME}/Ascend/nnae/latest
2. Run the following command to view the version information provided in the version field:
  1
  cat ascend_nnae_install.info

Installing the NNRT Offline Inference Engine Package

CANN NNRT offline inference engine package, which is installed in the edge inference scenario. It supports only offline inference and is mainly used for model inference of applications.

Add the execute permission on the software package and verify the consistency and integrity of the software package installation file.
1 2
chmod +x Ascend-cann-nnrt_<version>_linux-<arch>.run ./Ascend-cann-nnrt_<version>_linux-<arch>.run --check
If the following information is displayed, the software package verification is successful:
1
Verifying archive integrity... 100% SHA256 checksums are OK. All good.
Install the software package. (The installation command supports parameters such as --install-path= <path>. For details, see the parameter description.)
1
./Ascend-cann-nnrt_<version>_linux-<arch>.run --install
After the preceding command is executed, enter Y or y to agree to the Huawei Enterprise End User License Agreement (EULA) to continue the installation.
If the following information is displayed, the software is successfully installed:
1
xxx install success
xxx indicates the name of a software package to be installed.

If you do not specify an installation path, the software is installed in the default path. The default installation paths are as follows: For the root user, the path is /usr/local/Ascend. For a non-root user, the path is ${HOME} /Ascend. ${HOME} indicates the directory of the current user.
Configure environment variables. The default installation path of a non-root user is used as an example. Run the following command based on the actual path of set_env.sh:
1
source ${HOME}/Ascend/nnrt/set_env.sh
Verifying the installation. Run the following command to query the CANN version. If the query result is the same as the version of the installation software package, the installation is successful.
1. Go to the directory where the software package installation information file is stored. Replace it with the actual installation path. {arch} indicates the CPU architecture (aarch64 or x86_64).
  1
  cd ${HOME}/Ascend/nnrt/latest/<arch>-linux
2. Run the following command to view the version information provided in the version field:
  1
  cat ascend_nnrt_install.info

Installing the Kernels Operator Package

CANN binary operator package, including the dynamic and static library files for single-operator API execution (such as aclnn APIs) and kernel binary file.

Usage Scenarios:

This software package must be installed when single-operator APIs (such as aclnn APIs) are executed.
In the dynamic shape scenario in graph mode, you are advised to install this software package to improve the compilation performance.
In the static shape scenario in graph mode, if this software package is installed, the compilation performance can be improved. If this package is not installed, build a new kernel based on the specified shape to improve the operator execution performance.

Before installing the Kernels operator package, install the CANN software of the matching version and configure environment variables.

Add the execute permission on the software package and verify the consistency and integrity of the software package installation file.

chmod +x Ascend-cann-kernels-<chip_type>_<version>_linux-<arch>.run
./Ascend-cann-kernels-<chip_type>_<version>_linux-<arch>.run --check

If the following information is displayed, the software package verification is successful:

Verifying archive integrity...  100%   SHA256 checksums are OK. All good.

Run the corresponding command to install the software package based on the required file.
- To install all files, including the dynamic and static library files for single-operator API execution (such as aclnn APIs) and the kernel binary file, run the following command:
  1
  ./Ascend-cann-kernels-<chip_type>_<version>_linux-<arch>.run --devel
- To install files except the static library, run the following command:
  1
  ./Ascend-cann-kernels-<chip_type>_<version>_linux-<arch>.run --install
After the preceding command is executed, enter Y or y to agree to the Huawei Enterprise End User License Agreement (EULA) to continue the installation.
If the following information is displayed, the software is successfully installed:
1
xxx install success
xxx indicates the name of a software package to be installed.
Verifying the installation. Run the following command to query the software version. If the query result is the same as the version of the installation software package, the installation is successful.
1. Go to the directory where the software package installation information file is stored. Replace it with the actual installation path.
  1
  cd <path>/latest/opp_kernel
  Replace <path> with the installation path of the CANN package on which Kernels depends.
2. Run the following command to view the version information in the version_dir field:
  1
  cat version.info

(Optional) Installing the NNAL Library

The NNAL neural network acceleration library provides the Ascend Transformer Boost (ATB) acceleration library for foundation models, implements the neural network inference acceleration engine library based on the Transformer structure, and provides Ascend-affinity fusion operators, communication operators, and memory optimization, as the common foundation of operators, it improves the training and inference performance of foundation models.

Before installing the ATB, install Toolkit or NNAE of the matching version and configure environment variables.

Add the execute permission on the software package and verify the consistency and integrity of the software package installation file.
1 2
chmod +x Ascend-cann-nnal_<version>_linux-<arch>.run ./Ascend-cann-nnal_<version>_linux-<arch>.run --check
If the following information is displayed, the software package verification is successful:
1
Verifying archive integrity... 100% SHA256 checksums are OK. All good.
Install the software package. (The installation command supports parameters such as --install-path= <path>. For details, see the parameter description.)
1
./Ascend-cann-nnal_<version>_linux-<arch>.run --install
After the preceding command is executed, enter Y or y to agree to the Huawei Enterprise End User License Agreement (EULA) to continue the installation.
If the following information is displayed, the software is successfully installed:
1
xxx install success
xxx indicates the name of a software package to be installed.

If you do not specify an installation path, the software is installed in the default path. The default installation paths are as follows: For the root user, the path is /usr/local/Ascend. For a non-root user, the path is ${HOME} /Ascend. ${HOME} indicates the directory of the current user.
Configure environment variables. The default installation path of a non-root user is used as an example. Run the following command based on the actual path of set_env.sh:
1
source ${HOME}/Ascend/nnal/atb/set_env.sh
Configure the abi parameter when running the environment variable script.
- Automatic configuration: When the set_env.sh script is executed, if no parameter is added and the PyTorch environment is detected, the torch.compiled_with_cxx11_abi() API is automatically called, and the abi parameter during PyTorch compilation is selected as the abi parameter of the ATB. If the PyTorch environment is not detected, abi=1 is configured by default.
- Manual configuration: When running set_env.sh, you can specify the abi parameter of the ATB by setting --cxx_abi=1 and --cxx_abi=0.
Verifying the installation. Run the following command to query the NNAL software version. If the query result is the same as the version of the installation software package, the installation is successful.
1. Go to the directory where the software package installation information file is stored.
  1
  cd ${HOME}/Ascend/nnal/atb/latest
2. Run the following command to query the NNAL package version:
  1
  cat version.info

The glibc version must be 2.17 or later when you run the NNAL acceleration library. You can run the ldd --version command to query the glibc version (Mainstream Linux OSs meet the requirement of glibc versions. If the requirement is not meet, you are advised to reinstall the system of a new version. You are not advised to directly upgrade glibc, which may cause system breakdown).

Parent topic: Installing CANN (PM/VM Scenario)