Simple Custom Operator Project

Operator Project Creation

Similar to Operator Project Creation, a simple custom operator project is generated through msOpGen. The operator project is output based on the operator prototype definition, including the code implementation file on the host, implementation file on the kernel, and project compilation configuration file. The main difference is as follows: To create a simple operator project, you need to set the framework to aclnn by setting the -f option.

To create a simple operator development project using msOpGen, perform the following steps:

1. Compile the prototype definition JSON file of the operator to generate an operator development project.
   For example, the JSON file of the AddCustom operator is named add_custom.json. The file content is as follows:

   [
       {
           "op": "AddCustom",
           "input_desc": [
               {
                   "name": "x",
                   "param_type": "required",
                   "format": [
                       "ND",
                       "ND",
                       "ND"
                   ],
                   "type": [
                       "fp16",
                       "float",
                       "int32"
                   ]
               },
               {
                   "name": "y",
                   "param_type": "required",
                   "format": [
                       "ND",
                       "ND",
                       "ND"
                   ],
                   "type": [
                       "fp16",
                       "float",
                       "int32"
                   ]
               }
           ],
           "output_desc": [
               {
                   "name": "z",
                   "param_type": "required",
                   "format": [
                       "ND",
                       "ND",
                       "ND"
                   ],
                   "type": [
                       "fp16",
                       "float",
                       "int32"
                   ]
               }
           ]
       }
   ]

2. Use msOpGen to generate an operator development project. Taking the creation of the AddCustom operator project as an example, the following text explains only the key parameters. For details, see msOpGen.

   /python/site-packages/bin/msopgen gen -i $HOME/sample/add_custom.json -c ai_core- -lan cpp -out $HOME/sample/AddCustom -f aclnn

   • is the file storage path after the CANN package is installed. Replace it with the actual path.
   • -i: specifies the path of the operator prototype definition file add_custom.json. Change it to the actual path.
   • -c: ai_core- indicates that the operator is executed on the AI Core. indicates the model of the .
     

     The AI processor model <soc_version> can be obtained in the following ways:

     • For the following products: Run the npu-smi info command on the server where Ascend AI Processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.

       Atlas A2 training products / Atlas A2 inference products

       Atlas 200I/500 A2 inference products

       Atlas inference products

       Atlas training products

     • For the following products: Run the npu-smi info -t board -i id -c chip_id command on the server where Ascend AI Processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. Note that:
       • id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
       • chip_id: chip ID, which is obtained by running the npu-smi info -m command.

       Atlas A3 training products / Atlas A3 inference products

     Basic functions (operator development, compilation, and deployment based on the project) are applicable across operator projects created based on the same AI processor series.

   • -lan: cpp indicates that the operator is developed based on the Ascend C programming framework and using the C/C++ programming language.
   • -out: output path. The path can be either absolute or relative. The user who runs the tool must have the read and write permissions on the path. If this option is not specified, the outputs are generated to the current path where the command is executed.
   • -f: framework type. aclnn indicates that a simple project is generated.
3. After the command is executed, the operator project directory is generated in the specified directory or default path specified by -out. The project contains the operator implementation template file and compilation script. The following uses the AddCustom operator as an example. The directory structure is as follows:

   AddCustom
   ├── build.sh                        // Build entry point script
   ├── cmake 
   │   ├── config.cmake              // Compilation configuration item
   │   ├── func.cmake
   │   ├── intf.cmake
   │   └── util                       // Directory that stores the scripts used for operator project compilation and common build files
   ├── CMakeLists.txt                  // Build script of the operator project
   ├── op_host                         // Implementation file on the host
   │   ├── add_custom_tiling.h     // Operator tiling definition file.
   │   ├── add_custom.cpp          // Content file for operator prototype registration and tiling implementation
   │   ├── CMakeLists.txt
   ├── op_kernel                       // Implementation file on the kernel
   │   ├── CMakeLists.txt   
   │   ├── add_custom.cpp          // Operator code implementation file

   

   You simply need to pay attention to the files in bold during subsequent development.

Operator Implementation

Implement the operator by referring to Operator Implementation on the Kernel, Tiling Implementation on the Host, and Operator Prototype Definition.

Operator Building

After the operator is developed on the kernel and host sides, you need to compile the operator to generate a static operator library. The aclnn calling implementation code and header file are automatically generated, and the aclnn dynamic library is generated by linking the static operator library to support subsequent single-operator API calling (aclnn). Compilation process:

• The aclnn interface aclnn_*.h and aclnn implementation file aclnn_.cpp are automatically generated based on the operator implementation file on the host.
• Compile the tiling implementation and operator prototype definition to generate the tiling dynamic library liboptiling.so (libcust_opmaster_rt2.0).
• Compile the operator implementation file on the kernel side, load the dynamic tiling library, and generate the kernel static library libkernels.a.
• Compile the aclnn implementation file and link the kernel static library libkernels.a to generate the dynamic library libcust_opapi.so called by the single-operator API.

The following figure shows the compilation process.

Figure 1 Compilation process

The preceding process is encapsulated in the compilation script. To compile the script, perform the following steps:

1. Complete the project compilation configuration.
   • Modify the configuration options in the config.cmake file in the cmake directory. The content of the config.cmake file is as follows:

     set(CMAKE_CXX_FLAGS_DEBUG "")
     set(CMAKE_CXX_FLAGS_RELEASE "")
     
     if (NOT DEFINED CMAKE_BUILD_TYPE)
         set(CMAKE_BUILD_TYPE Release CACHE STRING "")
     endif()
     if (CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) 
         set(CMAKE_INSTALL_PREFIX "${CMAKE_SOURCE_DIR}/build_out" CACHE PATH "" FORCE)
     endif()
     if (NOT DEFINED ASCEND_CANN_PACKAGE_PATH)
         set(ASCEND_CANN_PACKAGE_PATH /usr/local/Ascend/cann CACHE PATH "") // Replace it with the actual path of the installed CANN software package.
     endif()
     if (NOT DEFINED ASCEND_PYTHON_EXECUTABLE)
         set(ASCEND_PYTHON_EXECUTABLE python3 CACHE STRING "")
     endif()
     if (NOT DEFINED ASCEND_COMPUTE_UNIT)
         set(ASCEND_COMPUTE_UNIT ascendxxx CACHE STRING "")
     endif()
     if (NOT DEFINED ENABLE_TEST)
         set(ENABLE_TEST FALSE CACHE BOOL "")
     endif()
     if (NOT DEFINED ENABLE_CROSS_COMPILE)
         set(ENABLE_CROSS_COMPILE  FALSE CACHE BOOL "")
     endif()
     if (NOT DEFINED CMAKE_CROSS_PLATFORM_COMPILER)
         set(CMAKE_CROSS_PLATFORM_COMPILER "/your/cross/compiler/path" CACHE PATH "")
     endif()
     set(ASCEND_TENSOR_COMPILER_PATH ${ASCEND_CANN_PACKAGE_PATH}/compiler)
     set(ASCEND_CCEC_COMPILER_PATH ${ASCEND_TENSOR_COMPILER_PATH}/ccec_compiler/bin)
     set(ASCEND_AUTOGEN_PATH ${CMAKE_BINARY_DIR}/autogen)
     file(MAKE_DIRECTORY ${ASCEND_AUTOGEN_PATH})
     set(CUSTOM_COMPILE_OPTIONS "custom_compile_options.ini")
     execute_process(COMMAND rm -rf ${ASCEND_AUTOGEN_PATH}/${CUSTOM_COMPILE_OPTIONS}
                     COMMAND touch ${ASCEND_AUTOGEN_PATH}/${CUSTOM_COMPILE_OPTIONS})

     Table 1 Common parameters to be configured

     Parameter	Description	Default Value
     ASCEND_CANN_PACKAGE_PATH	Installation path of the CANN package. Change it based on the actual situation.	"/usr/local/Ascend/cann"
     CMAKE_BUILD_TYPE	Compilation mode options: Release: release version, which does not contain debugging information. The final release version is compiled. Debug: debug version, which contains debugging information for you to develop and debug.	Release
     CMAKE_INSTALL_PREFIX	Directory for storing compilation results. If this parameter is not specified, the default value is used.	${CMAKE_SOURCE_DIR}/build_out: build_out directory in the operator project directory

   • (Optional) Configure environment variables for compilation.

     Table 2 Description of environment variables

     Environment Variable

     Configuration Description

     CMAKE_CXX_COMPILER_LAUNCHER

     Configures the C++ compiler (such as g++) and BiSheng Compiler launcher as ccache, enabling cache compilation, accelerating repeated compilation, and improving build efficiency. The following shows how to perform configuration in the corresponding CMakeLists.txt file: set(CMAKE_CXX_COMPILER_LAUNCHER <launcher_program>) <launcher_program> indicates the ccache installation path, for example, /usr/bin/ccache. set(CMAKE_CXX_COMPILER_LAUNCHER /usr/bin/ccache)

2. (Optional) If multiple operators need to be compiled, add the operators to be compiled to CMakeLists.txt in the op_kernel directory.

    
   # set custom compile options
   if ("${CMAKE_BUILD_TYPE}x" STREQUAL "Debugx")
       add_ops_compile_options(ALL OPTIONS -g -O0)
   endif()
   # For multi-operator compilation, run the add_kernel_compile command to add the operator source code file.
   add_kernel_compile(AddCustom ${CMAKE_CURRENT_SOURCE_DIR}/add_custom.cpp)
   add_kernel_compile(SubCustom ${CMAKE_CURRENT_SOURCE_DIR}/sub_custom.cpp)
   if(ENABLE_TEST AND EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/testcases)
       add_subdirectory(testcases)
   endif()

3. (Optional) In an operator project, if you want to add some customized compilation options to the kernel code of the operator, customize the compilation options by referring to Supported Customization Options.
4. Run the following command in the operator project directory to build the operator project:

   ./build.sh

   After the compilation is successful, the generated aclnn header file and library are stored in the CMAKE_INSTALL_PREFIX/op_api directory. Each operator corresponds to an independent header file. The directory structure is as follows:

   ├── op_api
   │   ├── include
   │       ├── aclnn_optype1.h
   │       └── aclnn_optype2.h
   │       └── aclnn_optypexxx.h
   │   ├── lib
   │       ├── libcust_opapi.so

   You can use the msobjdump tool to parse the library file generated in the lib directory to obtain the kernel information. For details, see msobjdump.

Single-Operator Execution

Complete single-operator API calling.