Dynamic and Static Operator Library Build

Operator Library Build

Dynamic and static operator library build is a process of generating dynamic and static libraries from operator implementation code and related files. Compared with custom operator package build, dynamic and static library build significantly simplifies the integration and deployment process. The process involves compiling and linking the operator implementation on the kernel, tiling implementation on the host, adaptation file for integration into a graph, and automatically generated single-operator call implementation code into dynamic and static libraries.

The following header files are automatically generated:

• aclnn header file for single-operator call, which is used in single-operator API call.
• Header file for operator prototype definition, which is used in operator integration into a graph.

The dynamic and static operator library build supports the following models:

• Atlas 350 Accelerator Card
• Atlas A3 training product/Atlas A3 inference product
• Atlas A2 training product/Atlas A2 inference product
• Atlas inference product

The procedure for building the dynamic and static operator libraries is as follows:

1. Complete the project compilation configuration.
   In addition to the aforementioned basic configuration, you need to set ASCEND_PACK_SHARED_LIBRARY to True in cacheVariables of CMakePresets.json in the project directory. The default value is False (a .run package is generated).

   "ASCEND_PACK_SHARED_LIBRARY": {
       "type": "BOOL",
       "value": "True"
    }

2. Run the following command in the operator project directory to build the operator project:

   ./build.sh

   After the build is successful, the following files are generated in the ${CMAKE_INSTALL_PREFIX}/op_api directory:

   • Header file for operator prototype definition, which is used in operator integration into a graph to define the operator prototype.
   • aclnn header file for single-operator call, which is used to provide APIs for directly calling operators.
   • Dynamic library libcust_opapi.so, which is used for dynamic linking.
   • Static library lib${vendor_name}.a, which is used for static linking.
   • Installation files ${vendor_name}-config.cmake and ${vendor_name}-targets.cmake, which are used to integrate the dynamic or static operator libraries of multiple vendors into a public dynamic library. ${vendor_name} represents the vendor name, and can also be regarded as the name of the operator package name generated by the same operator project. For details about the integration mode, see Operator Library Integration and Usage.

   The directory structure is as follows:

   ├── op_api
   │   ├── include
   │       ├── aclnn_optype1.h                  // aclnn header file for single-operator call
   │       └── aclnn_optype2.h
   │       └── aclnn_optypexxx.h
   │       └── op_proto.h                       // Header file for operator prototype definition
   │   ├── lib
   │       ├── cmake
   │         └── ${vendor_name}
   │            ├──${vendor_name}-config.cmake  // Installation file, which defines the configuration options and variables in the project
   │            └──${vendor_name}-targets.cmake // Installation file, which defines the build targets ${vendor_name}::static and ${vendor_name}::shared for static and dynamic libraries, respectively. This makes it easier to use the static and dynamic libraries and automatically handle library dependencies.
   │       ├── libcust_opapi.so                 // Dynamic operator library
   │       └── lib${vendor_name}.a              // Static operator library

Operator Library Integration and Usage

• Single-operator call

  In the single-operator call scenario, you can integrate and use the dynamic and static libraries generated in Operator Library Build as follows:

  For details, see the sample of integrating and using the static library.

  • Dynamic library integration

    find_package(${vendor_name} REQUIRED     # ${vendor_name} is the name of the generated operator package.
        PATHS ${CUST_PKG_PATH}               # ${CUST_PKG_PATH} is the path for storing the generated operator package.
        NO_DEFAULT_PATH
    )
    target_link_libraries(op_runner PRIVATE
        ...
        ${vendor_name}::shared               # The related target dependencies are automatically included.
        ...
    )

  • Static library integration

    find_package(${vendor_name} REQUIRED     # ${vendor_name} is the name of the generated operator package.
        PATHS ${CUST_PKG_PATH}               # ${CUST_PKG_PATH} is the path for storing the generated operator package.
        NO_DEFAULT_PATH
    )
    
    target_link_libraries(op_runner PRIVATE
        ...
        ${vendor_name}::static                # The related target dependencies are automatically included.
        ...
    )

  • Mixed use of static and dynamic libraries

    find_package(${vendor_name1} REQUIRED     # ${vendor_name1} is the name of the generated operator package.
        PATHS ${CUST_PKG_PATH_1}              # ${CUST_PKG_PATH_1} is the path for storing the generated operator package.
        NO_DEFAULT_PATH
    )
    find_package(${vendor_name2} REQUIRED     # ${vendor_name2} is the name of the generated operator package.
        PATHS ${CUST_PKG_PATH_2}              # ${CUST_PKG_PATH_2} is the path for storing the generated operator package.
        NO_DEFAULT_PATH
    )
    
    target_link_libraries(op_runner PRIVATE
        ...
        ${vendor_name1}::static                 # The related static library target dependencies are automatically included.
        ${vendor_name2}::shared                 # The related dynamic library target dependencies are automatically included.
        ...
    )

• The aforementioned path for storing the operator package is the ${CMAKE_INSTALL_PREFIX}/op_api directory of the operator project by default. You can copy the files in the op_api directory to your own directory and set ${CUST_PKG_PATH} to the custom directory.
• The names of the dynamic operator libraries generated by different operator projects are the same. To integrate multiple dynamic libraries, you need to change the name libcust_opapi.so.
  Before build, modify the CMakeLists.txt file in the op_host directory by adding the following code to set different output file names, in order to distinguish different dynamic libraries:

  set_target_properties(cust_opapi PROPERTIES
  	OUTPUT_NAME ${vendor_name}
  )

  After build, modify the ${vendor_name}-targets.cmake file in the op_api directory and change libcust_opapi.so to lib${vendor_name}.so. Ensure the consistency of the dynamic library names.

• Integrating an operator into a graph
  • Add the absolute path of dynamic library libcust_opapi.so to the ASCEND_CUSTOM_OPP_PATH environment variable. The GE framework searches for the dynamic operator library and uses it based on the environment variable during graph build and execution. The higher the environment variable is ranked, the higher the priority of the operator.

    export ASCEND_CUSTOM_OPP_PATH=${CMAKE_INSTALL_PREFIX}/op_api/lib/:${ASCEND_CUSTOM_OPP_PATH}

    

    If the dynamic library build and operator package build functions are used at the same time, the dynamic library generated by the former has a higher priority.

    In the following example, path1 and path3 are the directories generated during operator package build, and path2 and path4 are the directories for storing the dynamic library build artifacts. The build artifact priorities are 2 > 4 > 1 > 3.

    ASCEND_CUSTOM_OPP_PATH=<path1>/vendor_name1:<path2>/op_api/lib/:<path3>/vendor_name3:<path4>/op_api/lib/

  • If you use Ascend Graph to develop a graph, you can link the dynamic library libcust_opapi.so directly in the application build file, in addition to configuring environment variables. For details about Ascend Graph development, see the Graph Development. The .so file loading priority in dynamic library linking mode is higher than that in environment variable configuration mode.