Operator Dynamic and Static Library Compilation

Operator Library Compilation

Operator dynamic and static library compilation is the process of compiling the operator implementation code and related files into dynamic and static libraries. Compared with custom operator package compilation, dynamic and static library compilation significantly simplifies the integration and deployment process. This process includes compiling and linking the operator kernel implementation, tiling implementation on the host, graph adaptation file, and automatically generated single-operator calling implementation code into dynamic and static libraries.

The following header files are automatically generated:

• aclnn header file for single-operator calling, which is used in Single-Operator API Calling.
• Header file for operator prototype definition, which is used in Integrating Operators into a GE Graph.

The operator dynamic and static library compilation supports the following models:

• Atlas A3 training products/Atlas A3 inference products
• Atlas A2 training products/Atlas A2 inference products
• Atlas inference products

The procedure for compiling the operator dynamic and static libraries is as follows:

1. Complete the project compilation configuration.
   In addition to the basic configuration described above, you need to set the cacheVariables configuration item in the CMakePresets.json file in the project directory to True for the operator dynamic and static library compilation. The default value is False (a runfile will be generated).

   "ASCEND_PACK_SHARED_LIBRARY": {
       "type": "BOOL",
       "value": "True"
    }

2. Run the following command in the operator project directory to compile the operator project:

   ./build.sh

   After the compilation is successful, the following files are generated in the ${CMAKE_INSTALL_PREFIX}/op_api directory:

   • Header file for operator prototype definition, which is used in the operator graph scenario to define the operator prototype.
   • Header file aclnn for single-operator calling, which is used in the single-operator calling scenario to provide APIs for directly calling operators.
   • Dynamic library libcust_opapi.so, which is used for dynamic linking.
   • Static library lib${vendor_name}.a, which is used for static linking.
   • Installation files ${vendor_name}-config.cmake and ${vendor_name}-targets.cmake, which are used to integrate the operator dynamic or static libraries of multiple vendors into a public dynamic library. ${vendor_name} indicates the vendor name, or can be regarded as the name of the operator package generated by the same operator project. For details about the integration mode, see Operator Library Integration and Usage.

   The directory structure is as follows:

   ├── op_api
   │   ├── include
   │ ├── aclnn_optype1.h // Header file aclnn for single-operator calling.
   │       └── aclnn_optype2.h
   │       └── aclnn_optypexxx.h
   │ └── op_proto.h // Header file for defining the operator prototype
   │   ├── lib
   │       ├── cmake
   │         └── ${vendor_name}
   │ ├──${vendor_name}-config.cmake // installation file, which defines the configuration options and variables in the project.
   │ └──${vendor_name}-targets.cmake // installation file, which defines the build targets ${vendor_name}::static and ${vendor_name}::shared for static and dynamic libraries, respectively. This helps developers use static and dynamic libraries and automatically handle library dependencies.
   │ ├── libcust_opapi.so // Dynamic operator library
   │ └── lib${vendor_name}.a // Static operator library

Operator Library Integration and Usage

• Single-Operator Calling Scenario

  In the single-operator calling scenario, you can integrate and use the dynamic and static libraries generated in Operator Library Compilation as follows:

  For details about the complete example, see sample of integrating and using the static library.

  • Dynamic library integration

    find_package(${vendor_name} REQUIRED # ${vendor_name} is the name of the operator package generated after compilation.
        PATHS ${CUST_PKG_PATH} # ${CUST_PKG_PATH} is the path for storing the operator package generated after compilation.
        NO_DEFAULT_PATH
    )
    target_link_libraries(op_runner PRIVATE
        ...
        ${vendor_name}::shared # The related target dependencies are automatically included.
        ...
    )

  • Static library integration

    find_package(${vendor_name} REQUIRED # ${vendor_name} is the name of the operator package generated after compilation.
        PATHS ${CUST_PKG_PATH} # ${CUST_PKG_PATH} indicates the path for storing the operator package generated after compilation.
        NO_DEFAULT_PATH
    )
    
    target_link_libraries(op_runner PRIVATE
        ...
        ${vendor_name}::static # The related target dependencies are automatically included.
        ...
    )

  • Mixed use of static and dynamic libraries

    find_package(${vendor_name1} REQUIRED # ${vendor_name1} indicates the name of the operator package generated after compilation.
        PATHS ${CUST_PKG_PATH_1} # ${CUST_PKG_PATH_1} indicates the path for storing the operator package generated after compilation.
        NO_DEFAULT_PATH
    )
    find_package(${vendor_name2} REQUIRED # ${vendor_name2} indicates the name of the operator package generated after compilation.
        PATHS ${CUST_PKG_PATH_2} # ${CUST_PKG_PATH_2} indicates the path for storing the operator package generated after compilation.
        NO_DEFAULT_PATH
    )
    
    target_link_libraries(op_runner PRIVATE
        ...
        ${vendor_name1}::static # The related static library target dependencies are automatically included.
        ${vendor_name2}::shared # The related dynamic library target dependencies are automatically included.
        ...
    )

• The preceding path for storing the operator package is the ${CMAKE_INSTALL_PREFIX}/op_api directory of the operator project by default. You can copy the files in the op_api directory to your own directory. In this case, you can set ${CUST_PKG_PATH} to the custom directory.
• The names of the operator dynamic libraries generated by different operator projects are the same. If you need to integrate multiple dynamic libraries, you need to change the name of libcust_opapi.so.
  Before compilation, modify the CMakeLists.txt file in the op_host directory and add the following code to set different output file names to distinguish different dynamic libraries:

  set_target_properties(cust_opapi PROPERTIES
  	OUTPUT_NAME ${vendor_name}
  )

  After compilation, modify the ${vendor_name}-targets.cmake file in the generated op_api directory and change libcust_opapi.so to lib${vendor_name}.so. Ensure the consistency of the dynamic library names.

• Integrate an operator into a graph.
  • Set the ASCEND_CUSTOM_OPP_PATH environment variable to the absolute path of the dynamic library libcust_opapi.so. The GE framework searches for the operator dynamic library and uses it based on the environment variable during graph build and execution. The higher the environment variable is ranked, the higher the operator priority is.

    export ASCEND_CUSTOM_OPP_PATH=${CMAKE_INSTALL_PREFIX}/op_api/lib/:${ASCEND_CUSTOM_OPP_PATH}

    

    If the dynamic library compilation and operator package compilation functions are used at the same time, the dynamic library generated by the former has a higher priority.

    In the following example, path1 and path3 are the directories generated during operator package compilation, and path2 and path4 are the directories for storing the dynamic library compilation products. The compilation product priorities are 2 > 4 > 1 > 3.

    ASCEND_CUSTOM_OPP_PATH=<path1>/vendor_name1:<path2>/op_api/lib/:<path3>/vendor_name3:<path4>/op_api/lib/

  • If you use Ascend Graph to develop a graph, you can link the dynamic library libcust_opapi.so in the compilation file of the application in addition to configuring the environment variable. For details about Ascend Graph development, see Graph Mode Development Guide. The SO file loading priority in dynamic library link mode is higher than that in environment variable configuration mode.