Documentation

Compiling and Deploying Operators

Preparations

  • Compile the kernel code implementation file *.cpp of the Ascend C operator. The file can be released in source code or binary mode.
    • Source release: The kernel implementation of the operator is not compiled, and the kernel source code file *.cpp of the operator is retained. This method supports online operator compilation and operator compilation through ATC model conversion.
    • Binary release: Compile the kernel implementation of the operator and generate the JSON file *.json and operator binary file *.o that describe the operator information. If the operator binary needs to be directly called, use this compilation mode.
  • Compile the code implementation files *.cpp and *.h of the Ascend C operator on the host.
    • Compile the prototype definition and shape derivation implementation into the operator prototype definition dynamic library libcust_opsproto_*.so, and generate the external API op_proto.h of the operator prototype.
    • Compile the operator information library definition into the information library definition file *.json.
    • Compile the tiling implementation into a dynamic library, such as liboptiling.so.
    • Automatically generates the single-operator API calling code and header file aclnn_*.h, and compiles and generates the dynamic library libcust_opapi.so called by the single-operator API.

Compilation Process

After the operator kernel and host are developed, you need to build the operator project to generate the custom operator installation package *.run. For details about the build process, see Figure 1.

Figure 1 Compiling an operator project

Procedure

  1. Modify the cacheVariables configuration item of CMakePresets.json in the project directory to complete the project compilation configuration. The content of the CMakePresets.json file is as follows. For details about the parameters, see Table 1.
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
    		 
    {
    "version": 1,
    "cmakeMinimumRequired": {
        "major": 3,
        "minor": 19,
        "patch": 0
    },
    "configurePresets": [
        {
            "name": "default",
            "displayName": "Default Config",
            "description": "Default build using Unix Makefiles generator",
            "generator": "Unix Makefiles",
            "binaryDir": "${sourceDir}/build_out",
            "cacheVariables": {
                "CMAKE_BUILD_TYPE": {
                    "type": "STRING",
                    "value": "Release"
                },
                "ENABLE_SOURCE_PACKAGE": {
                    "type": "BOOL",
                    "value": "True"
                },
                "ENABLE_BINARY_PACKAGE": {
                    "type": "BOOL",
                    "value": "True"
                },
                "ASCEND_COMPUTE_UNIT": {
                    "type": "STRING",
                    "value": "ascendxxx"
                },
                "ENABLE_TEST": {
                    "type": "BOOL",
                    "value": "True"
                },
                "vendor_name": {
                    "type": "STRING",
                    "value": "customize"
                },
                "ASCEND_PYTHON_EXECUTABLE": {
                    "type": "STRING",
                    "value": "python3"
                },
                "CMAKE_INSTALL_PREFIX": {
                    "type": "PATH",
                    "value": "${sourceDir}/build_out"
                },
                "ENABLE_CROSS_COMPILE": {      // Enable cross compilation. Configure it based on the actual environment.
                    "type": "BOOL",
                    "value": "False"
                },
                "CMAKE_CROSS_PLATFORM_COMPILER": {     // Relace it with the actual path after the cross compilation tool is installed.
                    "type": "PATH",
                    "value": "/usr/bin/aarch64-linux-gnu-g++"
                }
            }
        }
    ]
}
    Table 1 Common parameters to be configured

    Parameter

    Description

    Default Value

    CMAKE_BUILD_TYPE

    Compilation mode options:

    • Release: release version, which does not contain debugging information. The final release version is compiled.
    • Debug: debug version, which contains debugging information for you to develop and debug.

    "Release"

    ENABLE_SOURCE_PACKAGE

    Whether to enable source code compilation

    "True"

    ENABLE_BINARY_PACKAGE

    Whether to enable binary compilation

    "True"

    vendor_name

    Indicates the name of the vendor to which the custom operator belongs. You are advised to specify the vendor name to avoid conflicts with OPPs provided by other vendors.

    "customize"
  2. Customize compilation options.
    Modify the CMakeLists.txt file in the op_kernel directory of the operator project and use add_ops_compile_options to add compilation options. 
    1
    		 
    add_ops_compile_options(OpType COMPUTE_UNIT soc_version1 soc_version2 ... OPTIONS option1 option2 ...)
    Table 2 Parameter description

    Parameter

    Optional/Required

    Description

    OpType

    Required

    The first parameter must be set to the operator type. To make the setting take effect for all operators in the operator project, set this parameter to ALL.

    COMPUTE_UNIT

    Optional

    AI processor models on which compilation options take effect. Multiple AI processor models are separated by spaces. If this parameter is not set, the configuration takes effect for all AI processor models.

    NOTE:

    The COMPUTE_UNIT configuration procedure is as follows:

    • Run the npu-smi info command on the server where the Ascend AI Processor is installed to obtain the Chip Name information. The actual value is AscendChip Name. For example, if Chip Name is xxxyy, the actual value is Ascendxxxyy. If Ascendxxxyy is the code sample path, you need to set ascendxxxyy.

    OPTIONS

    Required

    Custom compilation options. Multiple compilation options are separated by spaces.

    NOTE:
    • Add compilation options such as -sanitizer for debugging to enable msOpGen operator project compilation of the msSanitizer tool. 
      1
      		 
      add_ops_compile_options(ALL OPTIONS -sanitizer)
    • Add compilation options such as -g for debugging to enable the code call stack and hot spot map functions in the msprof op simulator scenario of the msProf tool. 
      1
      		 
      add_ops_compile_options(ALL COMPUTE_UNIT ascendxxxyy OPTIONS -g)
    • Add compilation options such as -g and -O0 for debugging to enable the msDebug tool. 
      1
      		 
      add_ops_compile_options(ALL OPTIONS -g -O0)
  3. Run the following command in the operator project directory to compile the operator project:
    ./build.sh

    After the compilation is successful, the build_out directory is created in the current directory, and the custom operator installation package custom_opp_ <target_os>_<target_architecture> .run is generated in the build_out directory.

    The operator implementation file name and kernel function name must be the same. They are the values after the operator type is converted using underscores (_).

    The following describes the process of converting the operator implementation file name and kernel function name through the operator type.
    • Replace the first uppercase letter with a lowercase letter. Example: Abc -> abc
    • If the character before an uppercase character is a lowercase character or digit, an underscore (_) is inserted before the uppercase character and the uppercase character is converted to a lowercase character. Example: AbcDef -> abc_def
    • If the character before an uppercase character is an uppercase character and the character after the uppercase character is a lowercase character, underscores (_) are inserted before the uppercase characters and the uppercase characters are converted to lowercase characters. Example: AbcAAc -> abc_a_ac.
    • Other uppercase characters are converted to lowercase characters, and lowercase characters remain unchanged.
  4. Deploy the OPP.

OPP Deployment

  1. Deploy the OPP.

    In the directory of the custom OPP, run the following command to install the OPP:

    ./custom_opp_ <target_os>_<target_architecture> .run --install-path = <path>, where --install-path is optional and is used to specify the installation directory of the custom operator package. An absolute path is supported. The running user must have the read and write permissions on the specified path.

    In the following description, <vendor_name> indicates the value of the vendor_name field in the CMakePresets.json configuration file during operator project compilation. The default value is customize.
    • In the default installation scenario, the --install-path parameter is not set. After the installation is successful, the custom operator files generated after building are deployed to the
      ${INSTALL_DIR}/opp/vendors/<vendor_name> directory. Replace ${INSTALL_DIR} with the actual CANN file directory. For example, if the Ascend-CANN-Toolkit software package is installed, the file path is $HOME/Ascend/ascend-toolkit/latest.

      The permission on the default installation path ${INSTALL_DIR}/opp/vendors of the custom operator package is related to the installation user and configuration of the CANN package. If the custom operator package fails to be installed due to insufficient permission, use the --install-path parameter and configure the environment variable ASCEND_CUSTOM_OPP_PATH to specify the installation directory (for details, see specifying an installation directory), or contact the installation user of the CANN package to modify the permission on the vendors directory. For details about cases, see Cannot Open the config.ini File During Operator Calling and Insufficient Permission Is Reported During OPP Deployment.

    • In an installation scenario with a specified directory, configure the --install-path parameter. After the installation is successful, the custom operator files generated after building are deployed to the <path>/vendors/<vendor_name> directory, and set_env.bash is added to the <path>/vendors/<vendor_name>/bin directory for setting environment variables related to the custom OPP.

      If the installation directory of the OPP is specified by configuring the --install-path parameter, run the source <path>/vendors/<vendor_name>/bin/set_env.bash command before using the custom operator. In the set_env.bash script, add the installation path of the custom OPP to the environment variable ASCEND_CUSTOM_OPP_PATH for the custom operator to take effect in the current environment.

    After the command is executed successfully, related files in the custom operator package are deployed in the current environment.

  2. You can view the directory structure after the deployment. The following example is based on the default installation scenario:
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    		 
    ├── opp    // Operator library directory
│   ├── vendors    // Directory of custom operators
│       ├── config.ini
│       └── vendor_name1   // Custom operator deployed by the storage vendor. vendor_name is configured during the compilation of the custom operator installation package. If vendor_name is not configured, the default value customize is used.
│           ├── framework     // Custom operator plugin library
│           ├── op_api
│           │   ├── include
│           │   │  └── aclnn_xx.h      // Declaration file of the operator calling API
│           │   └── lib
│           │       └── libcust_opapi.so
│           ├── op_impl
│           │   └── ai_core
│           │       └── tbe
│           │           ├── config
│           │           │   └── ${soc_version}     // Type of the Ascend AI Processor
│           │           │       └── aic-${soc_version}-ops-info.json     // Custom operator information library file
│           │           ├── vendor_name1_impl    // Code file for implementing the custom operator
│           │           │   └── dynamic
│           │           │       ├── xx.cpp
│           │           │       └── xx.py
│           │           ├── kernel     // Custom operator binary file
│           │           │   └── ${soc_version}     // Type of the Ascend AI Processor
│           │           │   └── config
│           │           └── op_tiling
│           │               ├── lib
│           │               └── liboptiling.so 
│           └── op_proto     // Directory of custom operator prototype library
│                ├── inc
│                │   └── op_proto.h
│                └── lib
│       ├── vendor_name2   // Custom operator deployed by storage vendor vendor_name2

    Argument: <soc_version>. To query the value, perform the following steps:

    • Run the npu-smi info command on the server where the Ascend AI Processor is installed to obtain the Chip Name information. The actual value is AscendChip Name. For example, if Chip Name is xxxyy, the actual value is Ascendxxxyy. If Ascendxxxyy is the code sample path, you need to set ascendxxxyy.
  3. Configure the priorities of custom operators.
    If multiple OPPs coexist and custom operators with the same OpType exist in different OPP directories, the operator in the OPP directory with the highest priority is used. The following describes how to configure the OPP priority.
    • Default installation scenario

      If custom operators of multiple vendors exist in the opp/vendors directory, you can configure the priority of the custom OPPs by configuring the config.ini file in the opp/vendors directory.

      The following provides a configuration template of config.ini.

      1
      		 
      load_priority=vendor_name1,vendor_name2,vendor_name3
      • load_priority: keyword of priority sequence configuration, which cannot be modified.
      • vendor_name1, vendor_name2, vendor_name3: priority sequence of custom operator vendors, which is arranged in descending order of priority.
    • Installation in a specified directory

      In an installation scenario with a specified directory, if multiple custom OPPs need to take effect, run the set_env.bash script in the installation directory of each OPP. Each time the script is executed, the installation directory of the current OPP is added to the beginning of the ASCEND_CUSTOM_OPP_PATH environment variable. Therefore, the priority can be determined based on the script execution sequence. A later script execution sequence indicates a higher priority of the corresponding OPP.

      For example, if source <path>/vendor_name1/bin/set_env.bash is executed before source <path>/vendor_name2/bin/set_env.bash, the priority of the vendor_name2 OPP is higher than that of the vendor_name1 OPP. The following is an example:

      ASCEND_CUSTOM_OPP_PATH=<path>/vendor_name2:<path>/vendor_name1:
    • The priority of the OPP installed in a specified directory is higher than that of the OPP installed in the default mode.
  4. Use the msOpST tool to test the operator kernel and verify the operator functions.
  5. Use the msSanitizer tool to detect operator memory and exceptions and locate operator accuracy exceptions.
  6. Use the msDebug tool to debug the operator on the board and further confirm the operator accuracy exception.
  7. Generates the compute memory heatmap, instruction pipeline chart, and operator instruction hot spot map based on the msProf tool, helping you further optimize operator performance.
  8. After finishing the preceding steps, ensure that the operator accuracy and performance meet the delivery requirements.