Operator Project Build Extension

When msOpGen is used to create an operator project, the related build scripts are fixed locally. If you need to use new features provided by the operator project (for example, support for MC2 operators), you must run msOpGen again to regenerate the project. To avoid frequent project reconstruction due to feature updates, the CMake script of the operator project is packed into the CANN software package. You can use find_package to search for the corresponding CMake modules and use the CMake function APIs exposed by the operator project.

In this development mode, you can organize your operator project by referring to the project directory below. The directory structure is similar to that generated by msOpGen. However, you do not need to create the CMake and scripts directories, as they are packed into the CANN software package. Take the AddCustom operator as an example. The directory structure is as follows:

AddCustom
├── CMakeLists.txt              // Top level of the operator project
├── CMakePresets.json           // Build configuration file, which is a built-in function of CMake. If this function is enabled, the CMake version must be 3.19 or later.
├── framework                   // Directory for storing the operator plugin implementation files during AI framework adaptation
├── op_host                     // Implementation file on the host
│   ├── add_custom.cpp         // File for operator prototype registration, shape inference, tiling implementation, and more
│   ├── add_custom_tiling.h    // Tiling definition file of the operator
│   └── CMakeLists.txt         // CMakeLists file on the host
├── op_kernel                   // Implementation file on the kernel
│   ├── Add
│   │   ├── add_custom.cpp    // Operator code implementation file
└   └── CMakeLists.txt         // CMakeLists file on the kernel

Compiling CMakeLists.txt

Top-level CMakeLists.txt of the operator project

Use find_package to find the corresponding build library.
Use npu_op_package to set the build artifact form of the operator project. The options are RUN, SHARED, and STATIC, which correspond to the .run package, dynamic library, and static library of the operator, respectively. This API can also be used to configure the content and installation location of the package (that is, the build artifact).
Add the subdirectories to be built.

cmake_minimum_required(VERSION 3.16.0)            # Specify the earliest CMake version required.
project(opp)
# 1. Use find_package to find the corresponding build library.
# ${ASCEND_CANN_PACKAGE_PATH} is the path of the corresponding CANN software package. You can directly use it.
find_package(ASC REQUIRED HINTS ${ASCEND_CANN_PACKAGE_PATH}/compiler/tikcpp/ascendc_kernel_cmake)
# 2. Use npu_op_package to set the build artifact form of the operator project.
set(package_name ${vendor_name})
npu_op_package(${package_name}                     # package name
                                                   # In RUN mode, the name is used as the directory name after the operator is deployed. In STATIC mode, the output is lib${package_name}.a.
    TYPE RUN                                       # Specify the build artifact form. The value can be RUN, STATIC, or SHARED.
    CONFIG
        ENABLE_SOURCE_PACKAGE True                 # Whether to pack the source code into the .run package.
        ENABLE_BINARY_PACKAGE True                 # Whether to build the kernel binary. In SHARED or STATIC mode, it must be set to True. The default value is True.
        INSTALL_PATH ${CMAKE_BINARY_DIR}/          # Installation location of the package.
)
# 3. Add the subdirectories to be built.
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/op_host)
    add_subdirectory(op_host)                      # Add the op_host subdirectory to the build.
endif()
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/op_kernel)
    add_subdirectory(op_kernel)                    # Add the op_kernel subdirectory to the build.
endif()

CMakeLists.txt on the host

Use npu_op_code_gen to generate the single-operator calling code for aclnn, the prototype definition code required for operator integration into a graph, and more.
In the single-operator calling scenario, use npu_op_library to build the aclnn single-operator calling library.
In the scenario of operator integration into a graph, use npu_op_library to build the operator prototype library required.
Use npu_op_library to build the library related to tiling.
Use npu_op_package_add to add the preceding libraries on the host to the corresponding package.

aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR} ops_srcs)
#1. Use npu_op_code_gen to generate the single-operator calling code for aclnn, the prototype definition code required for operator integration into a graph, and more.
npu_op_code_gen(
     SRC ${ops_srcs}                                                           # Collection of implementation files on the host.
     PACKAGE ${package_name}                                                   # Corresponds to package_name defined in npu_op_package.
     COMPILE_OPTIONS -g                                                        # Compilation options.
     OUT_DIR ${ASCEND_AUTOGEN_PATH}                                            # Path for storing the generated files.
)
# 2. In the single-operator calling scenario, use npu_op_library to build the aclnn single-operator calling library.
file(GLOB autogen_aclnn_src ${ASCEND_AUTOGEN_PATH}/aclnn_*.cpp)                # Add the aclnn source files generated through npu_op_code_gen to the source file list.
                                                                               
set_source_files_properties(${autogen_aclnn_src} PROPERTIES GENERATED TRUE)    # Set the file attribute to the file generated during the build process.
npu_op_library(cust_opapi ACLNN                                                # Specify the library name and type, as well as the source files involved in the build process.
     ${autogen_aclnn_src}
)
target_compile_options(cust_opapi PRIVATE                                      # Add compilation options to the library. This follows the standard CMake syntax.
     -fvisibility=hidden
)

#3. In the scenario of operator integration into a graph, use npu_op_library to build the operator prototype library required.
file(GLOB proto_src ${ASCEND_AUTOGEN_PATH}/op_proto.cc)                        # Add the source files related to the prototype definition generated through npu_op_code_gen to the source file list.
set_source_files_properties(${proto_src} PROPERTIES GENERATED TRUE)            # Set the file attribute to the file generated during the build process.
npu_op_library(cust_op_proto GRAPH                                             # Specify the library name and type, as well as the source files involved in the build process.
     ${ops_srcs}
     ${proto_src}
)
target_compile_options(cust_op_proto PRIVATE                                   # Add compilation options to the library. This follows the standard CMake syntax.
     -fvisibility=hidden
)

# 4. Use npu_op_library to build the library related to tiling.
file(GLOB fallback_src ${ASCEND_AUTOGEN_PATH}/fallback_*.cpp)                  # Add the tiling-related source files generated through npu_op_code_gen to the source file list.
                                                                               # This file is generated if EnableFallBack is configured for the operator.
set_source_files_properties(${fallback_src} PROPERTIES GENERATED TRUE)         # Set the file attribute to the file generated during the build process.
npu_op_library(cust_optiling  TILING                                           # Specify the library name and type, as well as the source files involved in the build process.
     ${ops_srcs}
     ${fallback_src}
)
target_compile_options(cust_optiling PRIVATE                                   # Add compilation options to the library. This follows the standard CMake syntax.
     -fvisibility=hidden
)

#5. Use npu_op_package_add to add the preceding libraries on the host to the corresponding package.
npu_op_package_add(${package_name}                                             # Corresponds to package_name defined in npu_op_package.
     LIBRARY                                                                   # Add the libraries on the host. The library names must be the same as those defined in the preceding steps.
          cust_optiling
          cust_opapi
          cust_op_proto
)

CMakeList.txt on the kernel

Use npu_op_kernel_options to add operator compilation options. For details about the supported compilation options, see the compilation option description.
Use npu_op_kernel_sources to specify the operator-specific directory and source files for compilation.
- If the source code files of the operator are not directly stored in the SRC_BASE directory (specified through npu_op_kernel_library), you can use KERNEL_DIR to specify a particular directory.
- If the name of the .cpp file for operator implementation on the kernel needs to be customized, you need to specify both OP_TYPE (operator type) and KERNEL_FILE (name of the .cpp file for kernel implementation) to configure the mapping between them. If they are not specified, the .cpp file name and the operator type must comply with the conversion rules.
Use npu_op_kernel_library to build the kernel library.
Use npu_op_package_add to add the preceding kernel library to the corresponding package.

# 1. Use npu_op_kernel_options to add operator compilation options.
npu_op_kernel_options(ascendc_kernels ALL OPTIONS --save-temp-files -g)# Add compilation options for the operator.

# 2. Use npu_op_kernel_sources to specify the operator-specific directory and source files for compilation.
npu_op_kernel_sources(ascendc_kernels # The name of the corresponding kernel library must be the same as that set through npu_op_kernel_library.
OP_TYPE AddCustom # If the name of the .cpp file for operator implementation on the kernel needs to be customized, configure the operator type.
KERNEL_DIR ./Add # Combine the KERNEL_DIR and SRC_BASE directories to form the directory where the .cpp file for kernel implementation is located.
COMPUTE_UNIT Ascendxxxyy # Set KERNEL_FILE to take effect on Ascendxxxyy.
KERNEL_FILE add_custom.cpp # If the name of the .cpp file for operator implementation on the kernel needs to be customized, configure the .cpp file name.
)

# 3. Use npu_op_kernel_library to build the kernel library.
npu_op_kernel_library(ascendc_kernels # Name of the kernel library.
SRC_BASE ${CMAKE_SOURCE_DIR}/op_kernel/ # Root directory of the source code on the kernel.
TILING_LIBRARY cust_optiling # Specify the dependent tiling library.
)
# 4. Add the kernel library to the package.
npu_op_package_add(${package_name} # Corresponds to package_name defined in npu_op_package.
LIBRARY ascendc_kernels # Add the kernel library. The library name must be the same as that defined in the preceding steps.
)

Build Environment Options

The following table lists the build environment options that can be configured and used.

Currently, options related to the Release and Debug versions are not supported.
Currently, cross-compilation options are not supported.

**Table 1** Build environment options
Type	Environment Option	Default Value	Description
STRING	vendor_name	customize	Name of the vendor to which the custom operator belongs. You are advised to specify the vendor name to avoid conflicts with OPPs provided by other vendors.
STRING	ASCEND_COMPUTE_UNIT	-	Ascend AI Processor model. The package of the corresponding model is built.
PATH	ASCEND_AUTOGEN_PATH	<CMAKE_BINARY_DIR>/autogen	Path for storing source files generated through npu_op_code_gen, such as the operator prototype library required for single-operator calling (aclnn file) and operator integration into a graph.
BOOL	ENABLE_SOURCE_PACKAGE	TRUE	Whether to enable source code compilation. If the source code and binary compilation configurations are set using npu_op_package, the configurations specified in npu_op_package have a higher priority.
BOOL	ENABLE_BINARY_PACKAGE	TRUE	Whether to enable binary compilation. If the package type is set to SHARED or STATIC, the value must be TRUE. If the source code and binary compilation configurations are set using npu_op_package, the configurations specified in npu_op_package have a higher priority.
PATH	ASCEND_CANN_PACKAGE_PATH	-	CANN software package path. By default, you do not need to set this option and can directly use it when compiling the CMake file. The following is an example path: /usr/local/Ascend/cann.

The preceding build environment options can be configured in either of the following ways:

Directly use the set command to configure the options.

# Directly use the command in CMakeLists.txt.
set(vendor_name "customize")
set(ASCEND_COMPUTE_UNIT "ascendxxxyy")

Use the CMakePresets.json file to configure the options. If you use this method, install CMake 3.19 or later.

{
    "version": 1,
    "cmakeMinimumRequired": {
        "major": 3,
        "minor": 19,
        "patch": 0
    },
    "configurePresets": [
        {
            "name": "default",
            "displayName": "Default Config",
            "description": "Default build using Unix Makefiles generator",
            "generator": "Unix Makefiles",
            "binaryDir": "${sourceDir}/build_out",
            "cacheVariables": {
                "ASCEND_COMPUTE_UNIT": {
                    "type": "STRING",
                    "value": "ascendxxxyy"
                },
                "vendor_name": {
                    "type": "STRING",
                    "value": "customize"
                }
            }
        }
    ]
}

Then, run the following command to perform build based on the preset specified through --preset.

// CMake build and compilation in the shell environment
cmake -S . -B build_out --preset=default // Search for CMakePresets.json in the current directory (.).

Build Command

If the package type is set to RUN, you can run the following build command and use CMake's packaging capability to generate the run package.

# Run the following command in the root directory of the project.
mkdir -p build_out
rm -rf build_out/*
cmake -S . -B build_out --preset=default
cmake --build build_out --target binary -j${nproc}
cmake --build build_out --target package -j${nproc}

If the package type is set to SHARED or STATIC, you can run the following build command:

# Run the following command in the root directory of the project.
mkdir -p build_out
rm -rf build_out/*
cmake -S . -B build_out --preset=default
cmake --build build_out --target binary -j${nproc}
cmake --build build_out --target install -j${nproc}

CMake Functions

The following table lists the available CMake functions.

Type	API	Description
package	npu_op_package	Creates a package.
package	npu_op_package_add	Adds a target or file to a package.
library	npu_op_library	Creates a host library.
	npu_op_kernel_library	Creates a kernel library.
	npu_op_kernel_options	Adds compilation options for the kernel target.
	npu_op_kernel_sources	Describes the source code information of the kernel target.
	npu_op_device_tiling_library	Creates a tiling library on the device.
Other	npu_op_code_gen	Executes the code generation process to generate the single-operator calling code for aclnn and the prototype definition code required for operator integration into a graph.

npu_op_package
Creates a package.
```
npu_op_package(<package_name> TYPE <type> [CONFIG] [ENABLE_SOURCE_PKG <value>] [ENABLE_BINARY_PACKAGE <value>] [INSTALL_PATH <path>])
```
The parameters are described as follows:
- <package_name>: (required) name of the package.
- TYPE <type>: (required) package type. The options are RUN, SHARED, and STATIC, which correspond to the .run package, dynamic library, and static library of the operator, respectively.
- [CONFIG]: (optional) Configures the content and installation location of the package.
  - [ENABLE_SOURCE_PKG <value>]: (optional) whether to package the source code into a package. The default value is True.
  - [ENABLE_BINARY_PACKAGE <value>]: (optional) whether to package binary files into a package. The default value is True.
  - [INSTALL_PATH <path>]: (optional) Specifies the installation path of the package. The default value is CMAKE_BINARY_DIR.
npu_op_package_add
Adds a target or file to a package.
```
# Add a target.
npu_op_package_add(<package_name> LIBRARY <target_name1> [<target_name2>...] ) 
# Add a file. It is supported only for .run packages.
npu_op_package_add(<package_name> FILES <file_name1> [<file_name2>...] [TYPE <target_type>] [PACKAGE_PATH <pkg_path>])
```
The parameters are described as follows:
- <package_name>: (required) name of the package.
- LIBRARY: (required) Specifies the name of the target to be added to the package.
  - <target_name1> [<target_name2>...]: (required) target name list.
- FILES: (required) Specifies the name of the file to be added to the package.
  - <file_name1> [<file_name2>...]: (required) file name list.
- [TYPE <target_type>]: (optional) Specifies the type of the file to be installed in the corresponding directory. The value can be ACLNN or GRAPH. If this parameter is set to ACLNN, the file is packaged to the directory containing the aclnn single-operator calling header file under the .run package directory. If this parameter is set to GRAPH, the file is packaged to the directory containing the prototype definition header file for operator integration into a graph under the .run package directory.
- [PACKAGE_PATH <pkg_path>]: (optional) Specifies the relative path of the file in the package. The TYPE and PACKAGE_PATH parameters are mutually exclusive. That is, you can only select one of them for configuration.
npu_op_library
Creates a host library.
```
npu_op_library(<library_name> TYPE <library_type> <files>) 
```
The parameters are described as follows:
- <library_name>: (required) name of the host library.
- TYPE <library_type>: (required) type of the host library. The value can be TILING, ACLNN, GRAPH, or TF_PLUGIN.
  - TILING: library related to tiling.
  - ACLNN: aclnn single-operator calling library.
  - GRAPH: operator prototype library required for operator integration into a graph.
  - TF_PLUGIN: library related to TensorFlow adaptation.
- <files>: (required) source files to be compiled.
npu_op_kernel_library
Creates a kernel library.
```
npu_op_kernel_library(<target_name> SRC_BASE <path> TILING_LIBRARY <tiling_target>) 
```
The parameters are described as follows:
- <target_name>: (required) name of the target.
- SRC_BASE <path>: (required) Specifies the base directory of the kernel source code. The value must be an absolute path, such as the absolute path of the op_kernel directory in the sample code.
- TILING_LIBRARY <tiling_target>: (required) Specifies the dependent tiling target.
npu_op_kernel_options
Adds compilation options for the kernel target.
```
npu_op_kernel_options(<target_name> <op_type> [COMPUTE_UNIT <soc_version>] OPTIONS ...)
```
The parameters are described as follows:
- <target_name>: (required) name of the target.
- <op_type>: (required) Defines the scope where the configuration takes effect. The value can be ALL or OP_TYPE. ALL indicates that the configuration takes effect for all operators, and OP_TYPE indicates that the configuration takes effect for specific operators.
- [COMPUTE_UNIT <soc_version>]: (optional) Sets the compilation options of the operator on a specific AI processor model. If this parameter is not set, the configuration takes effect for all models by default.
  The configuration of <soc_version> is as follows:
  - For the following products: Run the npu-smi info command on the server where Ascend AI Processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.
    Atlas A2 training products/Atlas A2 inference products
    
    Atlas 200I/500 A2 inference products
    
    Atlas inference products
    
    Atlas training products
  - For the following products: Run the npu-smi info -t board -i id -c chip_id command on the server where Ascend AI Processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. Note that:
    id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
    chip_id: chip ID, which is obtained by running the npu-smi info -m command.
    
    Atlas A3 training products/Atlas A3 inference products
- OPTION...: (required) compilation options passed to the compiler.
npu_op_kernel_sources
Describes the source code information of the kernel target, including the kernel implementation file and source code path of the operator.
```
npu_op_kernel_sources(<target_name> [OP_TYPE <op_type>] [KERNEL_DIR <path>] [COMPUTE_UNIT <soc_version>] [KERNEL_FILE <file>]) 
```
The parameters are described as follows:
- <target_name>: (required) name of the target.
- [OP_TYPE <op_type>]: (optional) operator type. This parameter must be used together with KERNEL_FILE.
- [KERNEL_DIR <path>]: (optional) Specifies the relative path of the kernel source code to SRC_BASE.
  If the source code files of the operator are not directly stored in the SRC_BASE directory (specified through npu_op_kernel_library), you can use KERNEL_DIR to specify a particular directory.
- [COMPUTE_UNIT <soc_version>]: (optional) Specifies the model on which the kernel file (defined through KERNEL_FILE) takes effect. By default, the kernel file takes effect on all models.
- [KERNEL_FILE <file>]: (optional) Specifies the name of the kernel implementation file that serves as the operator entrypoint.
  If the name of the .cpp file for operator implementation on the kernel needs to be customized, you need to specify both OP_TYPE (operator type) and KERNEL_FILE (name of the .cpp file for kernel implementation) to configure the mapping between them. If they are not specified, the .cpp file name and the operator type must comply with the conversion rules.
npu_op_device_tiling_library
Creates a tiling library on the device. When it is used, the package type can only be set to RUN.
```
npu_op_device_tiling_library(<target_name> <type> <files>) 
```
The parameters are described as follows:
- <target_name>: (required) name of the target.
- <type>: (required) type of the tiling artifact. The value can be SHARED or STATIC.
- <files>: (required) tiling source file.
npu_op_code_gen
Executes the code generation process to generate the single-operator calling code for aclnn and the prototype definition code required for operator integration into a graph.
```
npu_op_code_gen(SRC <src_files> OUT_DIR <output_dir> PACKAGE <pkg_name> [COMPILE_OPTIONS ...])
```
The parameters are described as follows:
- SRC <src_files>: (required) range of source files involved in code generation.
- OUT_DIR <output_dir>: (required) Specifies the output path of the generated code.
- PACKAGE <pkg_name>: (required) Specifies the name of the package for the generated code.
- [COMPILE_OPTIONS ...]: (optional) Specifies the compilation options during the compilation.

Parent topic: Project-based Operator Development