Documentation

Operator Package Deployment

Operator package deployment refers to the installation of the custom operator package. The build result of the operator project is automatically deployed to the operator package installation directory.

OPP Deployment

  1. Deploy the custom operator package.

    In the directory of the custom operator package, run the following command to install the OPP:

    ./custom_opp_<target os>_<target architecture>.run --install-path=<path>

    --install-path specifies the installation directory of the custom OPP. This parameter is optional. Absolute path is supported. The running user must have the read and write permissions on the specified path.

    In the following description, <vendor_name> indicates the value of the vendor_name field in the CMakePresets.json configuration file during operator project build. The default value is customize.
    • In the default installation scenario, the --install-path parameter is not set. After the installation is successful, the custom operator files generated after compilation are deployed to the ${INSTALL_DIR}/opp/vendors/<vendor_name> directory. Replace ${INSTALL_DIR} with the actual CANN file directory. For example, if the Ascend-CANN-Toolkit software package is installed, the file path is $HOME/Ascend/ascend-toolkit/latest.

      The permission on the default installation path ${INSTALL_DIR}/opp/vendors of the custom operator package is related to the installation user and configuration of the CANN package. If the custom operator package fails to be installed due to insufficient permission, use the --install-path parameter and configure the environment variable ASCEND_CUSTOM_OPP_PATH to specify the installation directory (for details, see Specifying an Installation Directory), or contact the installation user of the CANN package to modify the permission on the vendors directory. For details, see Cannot Open the config.ini File During Operator Calling and Insufficient Permission Is Reported During OPP Deployment.

    • In an installation scenario with a specified directory, configure the --install-path parameter. After the installation is successful, the custom operator files generated after building are deployed to the <path>/vendors/<vendor_name> directory, and set_env.bash is added to the <path>/<vendor_name>/bin directory for setting environment variables related to the custom OPP.

      If the installation directory of the OPP is specified by configuring the --install-path parameter, run the source <path>/vendors/<vendor_name>/bin/set_env.bash command before using the custom operator. In the set_env.bash script, add the installation path of the custom OPP to the environment variable ASCEND_CUSTOM_OPP_PATH for the custom operator to take effect in the current environment.

    After the command is executed successfully, related files in the custom operator package are deployed in the current environment.

  2. You can view the directory structure after the deployment. The following example is based on the default installation scenario:
    ├── opp    // Operator library directory
│   ├── vendors    // Directory of custom operators
│       ├── config.ini
│       └── vendor_name1   // Custom operator deployed by the storage vendor. vendor_name is configured during the build of the custom operator installation package. If vendor_name is not configured, the default value customize is used.
│           ├── framework     // Custom operator plugin library
│           ├── op_api
│           │   ├── include
│           │   │  └── aclnn_xx.h      // Declaration file of the operator calling API
│           │   └── lib
│           │       └── libcust_opapi.so
│           ├── op_impl
│           │   └── ai_core
│           │       └── tbe
│           │           ├── config
│           │           │   └── ${soc_version}     // Type of the Ascend AI Processor
│           │           │       └── aic-${soc_version}-ops-info.json     // Custom operator information library file
│           │           ├── vendor_name1_impl    // Code file for implementing the custom operator
│           │           │   └── dynamic
│           │           │       ├── xx.cpp
│           │           │       └── xx.py
│           │           ├── kernel     // Custom operator binary file
│           │           │   └── ${soc_version}     // Type of the Ascend AI Processor
│           │           │   └── config
│           │           └── op_tiling
│           │               ├── lib
│           │               └── liboptiling.so 
│           └── op_proto     // Directory of custom operator prototype library
│                ├── inc
│                │   └── op_proto.h
│                └── lib
│       ├── vendor_name2   // Custom operator deployed by storage vendor vendor_name2
  3. Configure the priorities of custom operators.
    If multiple OPPs coexist and custom operators with the same OpType exist in different OPP directories, the operator in the OPP directory with the highest priority is used. The following describes how to configure the OPP priority.
    • Default installation scenario

      If custom operators of multiple vendors exist in the opp/vendors directory, you can configure the priority of the custom OPPs by configuring the config.ini file in the opp/vendors directory.

      The following provides a configuration template of config.ini.

      load_priority=vendor_name1,vendor_name2,vendor_name3
      • load_priority: keyword of priority sequence configuration, which cannot be modified.
      • vendor_name1, vendor_name2, vendor_name3: priority sequence of custom operator vendors, which is arranged in descending order of priority.
    • Installation in a specified directory

      In an installation scenario with a specified directory, if multiple custom OPPs need to take effect, run the set_env.bash script in the installation directory of each OPP. Each time the script is executed, the installation directory of the current OPP is added to the beginning of the ASCEND_CUSTOM_OPP_PATH environment variable. Therefore, the priority can be determined based on the script execution sequence. A later script execution sequence indicates a higher priority of the corresponding OPP.

      For example, if source <path>/vendor_name1/bin/set_env.bash is executed before source <path>/vendor_name2/bin/set_env.bash, the priority of the vendor_name2 OPP is higher than that of the vendor_name1 OPP. The following is an example:

      ASCEND_CUSTOM_OPP_PATH=<path>/vendor_name2:<path>/vendor_name1:
    • The priority of the OPP installed in a specified directory is higher than that of the OPP installed in the default mode.

OPP Deployment Across Platforms

The custom operator package can be installed across platforms, and can be installed in the default path or a customized path.

Taking the default installation path as an example, install the AArch64 and x86_64 OPPs on the AArch64 platform. After the installation is successful, you can view that the directory structure is compatible with the two platforms, as shown in the following figure.

├── opp    // Operator library directory
│   ├── vendors    // Directory of custom operators
│       ├── config.ini
│       └── vendor_name1   // Custom operator deployed by the storage vendor. The vendor_name is configured during the build of the custom operator installation package. If vendor_name is not configured, the default value customize is used.
│           ├── framework     // Custom operator plugin library
│           ├── op_impl
│           │   └── ai_core
│           │       └── tbe
│           │           ├── config
│           │           │   └── ${soc_version} 
│           │           │       └── aic-${soc_version}-ops-info.json     // Custom operator information library file
│           │           ├── vendor_name1_impl    // Code file for implementing the custom operator
│           │           │   └── dynamic
│           │           │       ├── xx.cpp
│           │           │       └── xx.py
│           │           ├── kernel     // Custom operator binary file
│           │           │   └── ${soc_version}     
│           │           │   └── config
│           │           └── op_tiling
│           │               ├── lib
│           │               │   └──linux
│           │               │       ├──aarch64
│           │               │       │   └──libcust_opmaster_rt2.0.so
│           │               │       └──x86_64
│           │               │            └──libcust_opmaster_rt2.0.so
│           │               └── liboptiling.so  -> lib/linux/aarch64/libcust_opmaster_rt2.0.so
│           └── op_proto     // Directory of custom operator prototype library
│               ├── inc
│               │   └── op_proto.h
│               └── lib
│                    └──linux
│                         ├──aarch64
│                         │   └──libcust_opsproto_rt2.0.so
│                         └──x86_64
│                              └──libcust_opsproto_rt2.0.so
│       ├── vendor_name2   // Custom operator deployed by storage vendor vendor_name2
Parent topic: Build and Deployment