Documentation

Environment Setup

Software Package Installation

  1. Prepare software packages based on Table 1.
    Table 1 Software package list

    Component

    Description

    Toolkit

    Toolkit package for development, debugging, and tuning, which contains operator tools, model tools, and app tools.

    Firmware

    Firmware package, which basic firmware, such as UEFI, is usually burnt into during device production. Firmware can also be upgraded by installing this package in a later stage.

    Driver

    Driver package, which is used for interaction, scheduling, and transfer between the host and device. This package contains the device manager, query driver, graph execution task scheduling driver, training data transfer preprocessing driver, and AI CPU operator loading and execution driver.

    PyTorch (optional)

    When using the Python APIs of PyTorch for programming, you need to install PyTorch-related software.

    NOTE:

    You can click Gitee to download the latest PyTorch software from the Ascend/pytorch repository. Note that the recommended version is v5.0.rc2.1-pytorch1.11.0 or later. Otherwise, the JSON file capability of dump operators will be affected.
  2. Install the software package.

    Install the driver, firmware, Ascend-CANN-Toolkit, and the optional PyTorch AI framework. For details, see the CANN Software Installation Guide.

  3. Set the environment variables. The operator compilation tool depends on the AOE. Therefore, you need to configure the basic environment variables of the CANN software and the environment variables required by the AOE tool.
    • Basic environment variables of the CANN software

      The CANN portfolio provides a process-level environment variable setting script to automatically set environment variables. The following commands are used as examples, in which the default installation paths are under the root or non-root user. Replace them with actual installation paths.

      # Install Toolkit as the root user.
. /usr/local/Ascend/ascend-toolkit/set_env.sh 
# Install Toolkit as a non-root user.
. ${HOME}/Ascend/ascend-toolkit/set_env.sh
    • AOE depends on Python. Take Python 3.7.5 as an example. Run the following commands as the running user to configure the environment variables related to Python 3.7.5:
      # Set the Python 3.7.5 library path.
export LD_LIBRARY_PATH=/usr/local/python3.7.5/lib:$LD_LIBRARY_PATH
# If multiple Python 3 versions exist in the user environment, use Python 3.7.5.
export PATH=/usr/local/python3.7.5/bin:$PATH

      Replace the Python 3.7.5 installation path based on the actual requirements. You can also write the preceding commands to the ~/.bashrc file and run the source ~/.bashrc command to make the modification take effect immediately.

    • Before tuning, you can configure other optional environment variables by referring to the following example. For details, see Table 2.
      export ASCEND_DEVICE_ID=0
export TUNE_BANK_PATH=/home/HwHiAiUser/custom_tune_bank
export TE_PARALLEL_COMPILER=8
export REPEAT_TUNE=False

      You can write the commands for configuring environment variables to the custom script for future use.

      Table 2 Environment variables

      Environment Variable

      Description

      ASCEND_DEVICE_ID

      Logical ID of the Ascend AI Processor.

      The value range is [0, N – 1] and the default value is 0. N indicates the number of devices on the physical machine, VM, or in a container.

      TUNE_BANK_PATH

      Path of the custom repository generated after tuning.

      The path must be an absolute path or a relative path to the path of the AOE tuning engine. The path must exist and the user must have the read, write, and execute permissions on the path. If the path specified by TUNE_BANK_PATH does not exist or the user does not have the required permissions on the path, the tuning process will report an error and exit.

      The priority of the paths for storing the custom repository is: TUNE_BANK_PATH > ASCEND_CACHE_PATH > default path. For details about TUNE_BANK_PATH and ASCEND_CACHE_PATH, see the Environment Variables.
      • Custom operator repository
        • If this environment variable is not configured, run the env command to check whether ASCEND_CACHE_PATH exists. If it exists, the custom repository is stored in ${ASCEND_CACHE_PATH}/aoe_data/${soc_version}. If it does not exist, the custom repository is stored in ${HOME}/Ascend/latest/data/aoe/custom/op/${soc_version} by default.
        • If this environment variable is configured, the optimal policy after tuning is stored in ${soc_version} in the configured path.
      NOTE:

      If multiple users share the repository, the users must set TUNE_BANK_PATH to the same path and have the read and write permissions on the path.

      If the repository path is customized before tuning, you also need to configure this environment variable if you want to use the custom repository during model conversion.

      TE_PARALLEL_COMPILER

      Environment variable required for operator build.

      Parallel build is especially useful when a deep network is to build.

      The value of TE_PARALLEL_COMPILER indicates the number of operator build processes, which must be an integer. If the value is greater than 1, parallel build is enabled. In a scenario where AOE tuning is enabled, the maximum value of the environment variable is calculated as follows: Maximum value = Number of CPU cores x 80%/Number of Ascend AI Processors. The value ranges from 1 to 32. The default value is 8.

      This environment variable can accelerate operator build. Therefore, it can accelerate the tuning of processes related to operator build.

      REPEAT_TUNE

      Whether to initiate tuning again. This environment variable takes effect only when subgraph tuning or operator tuning is enabled.

      If it is set to False and a network tuning case (a tiling policy for a specific shape) is available in the repository (built-in or custom), the tuning process of the case is skipped. When the logic of some operators is changed, for example, the ND input support is added to the GEMM operator, you need to set this environment variable to True and initiate tuning again.

      The value can be True or False. The default value is False.

Tool Acquisition

The tool is stored in the ${INSTALL_DIR} /compiler/bin/op_compiler directory.

Replace ${INSTALL_DIR} with the actual CANN component directory. If the Ascend-CANN-Toolkit package is installed as the root user, the CANN component directory is /usr/local/Ascend/ascend-toolkit/latest.