Operator Project Build

After the operator kernel and host implementations are developed, build the operator project to generate a custom OPP runfile (*.run). The build workflow includes:

  • Compile the code implementation file *.cpp on the kernel of the Ascend C operator, which can be released in source code or binary mode.
    • Source release: The kernel implementation of the operator is not compiled, and the kernel source code file *.cpp of the operator is retained. This method supports online operator build and operator build through ATC model conversion.
    • Binary release: Compile the kernel implementation of the operator and generate the JSON file *.json and operator binary file *.o that describe the operator information. If the operator binary needs to be directly called, use this compilation mode. For example, call a single-operator in Single-Operator API Calling mode, call a single-operator in the PyTorch framework, or call an operator in a dynamic network.
  • Compile the code implementation files *.cpp and *.h of the Ascend C operator on the host.
    • Compile the prototype definition and shape derivation implementation into the operator prototype definition dynamic library libcust_opsproto_*.so, and generate the external interface op_proto.h of the operator prototype.
    • Compile the Tiling implementation into the Tiling dynamic library liboptiling.so.
    • The single-operator API call code and header file aclnn_*.h are automatically generated based on the operator prototype definition, and the dynamic library libcust_opapi.so for single-operator API call is generated through compilation.
The following figure shows the build process.
Figure 1 Compiling an operator project

Procedure

  1. Complete the project build configuration.
    • Modify cacheVariables in CMakePresets.json under the project directory. The content of the CMakePresets.json file is as follows. For details about the parameters to be configured, see Table 1. Other parameters are automatically generated during project creation.
      {
          "version": 1,
          "cmakeMinimumRequired": {
              "major": 3,
              "minor": 19,
              "patch": 0
          },
          "configurePresets": [
              {
                  "name": "default",
                  "displayName": "Default Config",
                  "description": "Default build using Unix Makefiles generator",
                  "generator": "Unix Makefiles",
                  "binaryDir": "${sourceDir}/build_out",
                  "cacheVariables": {
                      "CMAKE_BUILD_TYPE": {
                          "type": "STRING",
                          "value": "Release"
                      },
                      "ENABLE_SOURCE_PACKAGE": {
                          "type": "BOOL",
                          "value": "True"
                      },
                      "ENABLE_BINARY_PACKAGE": {
                          "type": "BOOL",
                          "value": "True"
                      },
                      "ASCEND_COMPUTE_UNIT": {
                          "type": "STRING",
                          "value": "ascendxxx"
                      },
                      "ENABLE_TEST": {
                          "type": "BOOL",
                          "value": "True"
                      },
                      "vendor_name": {
                          "type": "STRING",
                          "value": "customize"
                      },
                      "ASCEND_PYTHON_EXECUTABLE": {
                          "type": "STRING",
                          "value": "python3"
                      },
                      "CMAKE_INSTALL_PREFIX": {
                          "type": "PATH",
                          "value": "${sourceDir}/build_out"
                      },
                      "ENABLE_CROSS_COMPILE": {
                          "type": "BOOL",
                          "value": "False"
                      },
                      "CMAKE_CROSS_PLATFORM_COMPILER": {
                          "type": "PATH",
                          "value": "/usr/bin/aarch64-linux-gnu-g++"
                      },
                      "ASCEND_PACK_SHARED_LIBRARY": {
                          "type": "BOOL",
                          "value": "False"
                      }
                  }
              }
          ]
      }
      Table 1 Parameters to be configured

      Parameter

      Description

      Default Value

      CMAKE_BUILD_TYPE

      Compilation mode options:

      • Release: release version, which does not contain debugging information. The final release version is compiled.
      • Debug: debug version, which contains debugging information for you to develop and debug.

      Release

      ENABLE_SOURCE_PACKAGE

      Whether to enable source code compilation

      True: enabled.

      ENABLE_BINARY_PACKAGE

      Whether to enable binary compilation

      True: enabled.

      vendor_name

      Indicates the name of the vendor to which the custom operator belongs. You are advised to specify the vendor name to avoid conflicts with OPPs provided by other vendors.

      customize

      ASCEND_PACK_SHARED_LIBRARY

      Whether to enable dynamic library compilation

      False

    • (Optional) Configure environment variables for compilation.
      Table 2 Description of environment variables

      Environment Variable

      Configuration Description

      CMAKE_CXX_COMPILER_LAUNCHER

      Configures the C++ compiler (such as g++) and BiSheng Compiler launcher as ccache, enabling cache compilation, accelerating repeated compilation, and improving build efficiency. Before using this function, you need to install ccache.

      The following shows how to perform configuration in the corresponding CMakeLists.txt file:

      set(CMAKE_CXX_COMPILER_LAUNCHER <launcher_program>)

      <launcher_program> indicates the ccache installation path, for example, /usr/bin/ccache.

      set(CMAKE_CXX_COMPILER_LAUNCHER /usr/bin/ccache)
  2. Run the following command in the operator project directory to compile the operator project:

    ./build.sh

    After the project is built successfully, an OPP runfile named custom_opp_<target os>_<target architecture>.run is generated in the build_out directory.

    To save build process logs, use the environment variable ASCENDC_BUILD_LOG_DIR to control the storage path. If no error occurs during compilation, the log file name is followed by _success. If an error occurs, the corresponding error information is displayed on the screen, indicating the path and name of the log file, and the log file name is followed by _error.

    # To store build logs in /home/build_log/, do as follows and disable log storage by default.
    export ASCENDC_BUILD_LOG_DIR=/home/build_log/

Cross Compilation of OPP

After the operator code is implemented, if the current platform architecture is consistent with the operating environment, compile the operator package by referring to the previous section. To implement cross compilation of the operator package, perform the following steps:

  1. Download the cross compilation tool. The following table uses Ubuntu as an example to describe how to download the compilation tool. For other operating systems, replace it with the actual download command.
    Table 3 Example command for downloading the cross compilation tool for Ubuntu

    Current Platform Architecture

    Operating Environment Platform Architecture

    Commands for Downloading Build Tools

    x86_64

    aarch64

    sudo apt-get install -y g++-aarch64-linux-gnu

    aarch64

    x86_64

    sudo apt-get install g++-x86-64-linux-gnu

  2. Cross compile the custom operator project and generate the custom operator package.
    1. Change the value of ENABLE_CROSS_COMPILE in CMakePresets.json to True to enable cross compilation.
      "ENABLE_CROSS_COMPILE": {
          "type": "BOOL",
          "value": "True"
       }
    2. Change CMAKE_CROSS_PLATFORM_COMPILER in CMakePresets.json to the path of the cross compilation tool after installation.
      "CMAKE_CROSS_PLATFORM_COMPILER": {
          "type": "PATH",
          "value": "/usr/bin/aarch64-linux-gnu-g++"
      }
    3. Run the following command in the operator project directory to cross compile the operator project:

      ./build.sh

      After the project is built successfully, an OPP runfile named custom_opp_<target os>_<target architecture>.run is generated in the build_out directory.

Supported Customization Options

In an operator project, if you want to add some custom compilation options to the kernel code, you can customize the compilation options by referring to the following method:

Modify the CMakeLists.txt file in the op_kernel directory of the operator project and use add_ops_compile_options to add compilation options as follows:

add_ops_compile_options(OpType COMPUTE_UNIT soc_version1 soc_version2 ... OPTIONS option1 option2 ...)

The following table describes the parameters.

Table 4 Parameter Description

Parameter

Optional/Required

Description

OpType: operator type.

Required

The first parameter must be set to the operator type. To make the setting take effect for all operators in the operator project, set this parameter to ALL.

COMPUTE_UNIT

Optional

AI processor models on which compilation options take effect. Multiple AI processor models are separated by spaces. If this parameter is not set, the configuration takes effect for all AI processor models.

NOTE:

The COMPUTE_UNIT configuration procedure is as follows:

  • For the following products: Run the npu-smi info command on the server where AI processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.

    Atlas A2 training product / Atlas A2 inference product

    Atlas 200I/500 A2 inference product

    Atlas inference product

    Atlas training product

  • For the Atlas A3 training product / Atlas A3 inference product : Run the npu-smi info -t board -i id -c chip_id command on the server where AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. Note that:
    • id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
    • chip_id: chip ID, which is obtained by running the npu-smi info -m command.
  • For the Atlas 350 Accelerator Card: Run the npu-smi info -t board -i id command on the server where AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234.

    id indicates the device ID, which is the NPU ID obtained by running the npu-smi info -l command.

OPTIONS

Required

Custom compilation options. Multiple compilation options are separated by spaces.

  • Adds the -D compilation option to define macros during compilation.
    OPTIONS -Dname=definition
  • Adds compilation options such as -g and -O0 for debugging.
  • Supports passing BiSheng Compiler options:

    For example, --cce-auto-sync=off can be used to disable the automatic synchronization function. This option is enabled by default in custom operator projects and does not need to be manually configured. For details, see Automatic Synchronization.

    For more compilation options, see BiSheng Compiler options.

  • The compilation options provided by the Ascend C framework are described in the following sections.
  • Compilation options are configured based on the operator type and AI processor model series. That is, different compilation options can be configured for different operator types and AI processor model series.
    add_ops_compile_options(AddCustom COMPUTE_UNIT Ascendxxxyy ... OPTIONS -DNEW_MACRO1=xx)
    add_ops_compile_options(AddCustom COMPUTE_UNIT Ascendxxxyy ... OPTIONS -DNEW_MACRO2=xx)
    add_ops_compile_options(AddCustom COMPUTE_UNIT Ascendxxxyy ... OPTIONS -DNEW_MACRO3=xx)
  • For the same operator type and AI processor model series, configure compilation options for multiple times. The latest configuration is used.
  • If there is no conflict between the compilation options that take effect for ALL and the compilation options that take effect for a single-operator, the compilation options take effect at the same time. If there is a conflict, use the compilation options of the single-operator.
The compilation options provided by the Ascend C framework are described as follows:
  • --tiling_key: After this option is set, only the kernel code related to the specified TilingKey is compiled to accelerate compilation. If no TilingKey is specified, all TilingKeys are compiled by default. When multiple TilingKeys are configured, no space is allowed between them. In the following example, 1 and 2 are tiling_keys.
    --tiling_key=1,2
  • For details about the compilation macro switches, see Built-in Compilation Macro Switches.
  • --op_relocatable_kernel_binary: If this option is set to true, an additional binary file that can be re-linked is compiled. If this option is not set or is set to false, this option does not take effect. This option is used to for the scenario where SuperKernel is enabled for the custom tiling offload operator. The binary file generated by configuring this option can be directly reused during SuperKernel compilation of the operator, reducing the compilation time.
  • --kernel-template-input: This option is used for operator project template programming. After this option is set, only the kernel code related to the specified template parameter combination is compiled, accelerating the compilation process. If this option is not set, all template parameter combinations are compiled by default. The input parameters are key-value pairs, which must be enclosed in double quotation marks ("") or single quotation marks (''). Different template parameters are separated by semicolons (;). Multiple values of the same template parameter are separated by commas (,). No space is allowed during configuration. The following is an example:
    --kernel-template-input="D_T_X=A1,A2;D_T_Y=B;D_T_Z=C"
    --kernel-template-input='D_T_X=A1,A2;D_T_Y=B;D_T_Z=C'

    When configuring template parameter combinations, ensure that the template parameter names match those defined at the kernel entry and on the host. For the values of template parameter combinations, if there are custom types, replace them with their corresponding numeric values. If the data type is natively supported, the value must be the same as the input parameter at the kernel entry. The following is an example:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    // Template parameter definition
    #define ADD_TPL_FP16 10
    #define ADD_TPL_FP32 20
    
    ASCENDC_TPL_ARGS_DECL(AddCustomTemplateNativeDtype,
        ASCENDC_TPL_DATATYPE_DECL(D_T_X, C_DT_FLOAT, C_DT_FLOAT16, ASCENDC_TPL_INPUT(0)),
        ASCENDC_TPL_DTYPE_DECL(D_T_Y, ADD_TPL_FP16, ADD_TPL_FP32),
        ASCENDC_TPL_DATATYPE_DECL(D_T_Z, C_DT_FLOAT, C_DT_FLOAT16, ASCENDC_TPL_OUTPUT(0)),
        ASCENDC_TPL_UINT_DECL(TILE_NUM, ASCENDC_TPL_8_BW, ASCENDC_TPL_UI_MIX, 2, 0, 2, 3, 5, 10, 12, 13, 9, 8),
        ASCENDC_TPL_BOOL_DECL(IS_SPLIT, 0, 1),
    );
    
    ASCENDC_TPL_SEL(
        ASCENDC_TPL_ARGS_SEL(
        ASCENDC_TPL_DATATYPE_SEL(D_T_X, C_DT_FLOAT),
        ASCENDC_TPL_DTYPE_SEL(D_T_Y, ADD_TPL_FP32),
        ASCENDC_TPL_DATATYPE_SEL(D_T_Z, C_DT_FLOAT),
        ASCENDC_TPL_UINT_SEL(TILE_NUM, ASCENDC_TPL_UI_LIST, 1, 8),
        ASCENDC_TPL_BOOL_SEL(IS_SPLIT, 0, 1),
        ASCENDC_TPL_DETERMINISTIC_SEL(true),
        ASCENDC_TPL_KERNEL_TYPE_SEL(ASCENDC_TPL_AIV_ONLY),
        ),
        ASCENDC_TPL_ARGS_SEL(
        ASCENDC_TPL_DATATYPE_SEL(D_T_X, C_DT_FLOAT16),
        ASCENDC_TPL_DTYPE_SEL(D_T_Y, ADD_TPL_FP16),
        ASCENDC_TPL_DATATYPE_SEL(D_T_Z, C_DT_FLOAT16),
        ASCENDC_TPL_UINT_SEL(TILE_NUM, ASCENDC_TPL_UI_LIST, 1, 8),
        ASCENDC_TPL_BOOL_SEL(IS_SPLIT, 0, 1),
        ASCENDC_TPL_DETERMINISTIC_SEL(false),
        ASCENDC_TPL_KERNEL_TYPE_SEL(ASCENDC_TPL_AIV_ONLY),
        ),
    );
    #endif
    
    // Kernel entry
    if constexpr (std::is_same_v<D_T_X, float> && std::is_same_v<D_T_Z, float>) {
            KernelAdd<D_T_X, float, D_T_Z> op;
            op.Init(x, y, z, tiling_data.totalLength, TILE_NUM);
            op.Process1();
        } else if constexpr (std::is_same_v<D_T_X, half> && std::is_same_v<D_T_Z, half>){
            KernelAdd<D_T_X, half, D_T_Z> op;
            op.Init(x, y, z, tiling_data.totalLength, TILE_NUM);
    

    The compilation option example is as follows:

    --kernel-template-input="D_T_X=10;D_T_Y=half;D_T_Z=10" 
    --kernel-template-input="D_T_X=10,20;D_T_Y=half,float;D_T_Z=10,20"