The CANN development kit provides the custom operator project generation tool msOpGen, which can generate operator projects based on the operator prototype definition. An operator project includes the operator code file for implementation on the host, operator implementation file on the kernel, and project compilation configuration file.
Before using msOpGen to create the operator project, install the driver firmware and CANN package by referring to Environment Setup to set up the development environment and operating environment.
To create an operator development project using msOpGen, perform the following steps:
- Compile the prototype definition JSON file of the operator to generate an operator development project. For details about the parameters in the JSON file, see Operator Development Tools.For example, the JSON file of the AddCustom operator is named add_custom.json. The file content is as follows:
[ { "op": "AddCustom", "input_desc": [ { "name": "x", "param_type": "required", "format": [ "ND", "ND", "ND" ], "type": [ "float16", "float", "int32" ] }, { "name": "y", "param_type": "required", "format": [ "ND", "ND", "ND" ], "type": [ "float16", "float", "int32" ] } ], "output_desc": [ { "name": "z", "param_type": "required", "format": [ "ND", "ND", "ND" ], "type": [ "float16", "float", "int32" ] } ] } ]For example, the JSON file of the ReduceMaxCustom operator (including attributes) is named reduce_max_custom.json. The file content is as follows:[ { "op": "ReduceMaxCustom", "input_desc": [ { "name": "x", "param_type": "required", "format": ["ND"], "type": ["float16"] } ], "output_desc": [ { "name": "y", "param_type": "required", "format": ["ND"], "type": ["float16"] }, { "name": "idx", "param_type": "required", "format": ["ND"], "type": ["int32"] } ], "attr": [ { "name": "reduceDim", "param_type": "required", "type": "int" }, { "name": "isKeepDim", "param_type": "optional", "type": "int", "default_value": 1 } ] } ] The value of the field op in the prototype definition JSON file must use upper camel case, where uppercase letters distinguish different semantic parts.
- Generate a development project of the operator using msOpGen. The following uses the AddCustom operator project as an example to describe how to generate an operator project. The following describes only key parameters. For details about the parameters, see Operator Development Tools.
${INSTALL_DIR}/python/site-packages/bin/msopgen gen -i $HOME/sample/add_custom.json -c ai_core-<soc_version> -lan cpp -out $HOME/sample/AddCustom
- ${INSTALL_DIR} is the file storage path after the CANN software is installed. Replace it with the actual path.
- -i: specifies the path of the operator prototype definition file add_custom.json. Change it to the actual path.
- -c: ai_core-<soc_version> indicates that the operator is executed on the AI Core. <soc_version> indicates the model of the AI processor.
The AI processor model <soc_version> can be obtained in the following ways:
- For the following products: Run the npu-smi info command on the server where AI processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.
Atlas A2 training product /Atlas A2 inference product Atlas 200I/500 A2 inference product Atlas inference product Atlas training product - For the
Atlas A3 training product /Atlas A3 inference product : Run the npu-smi info -t board -i id -c chip_id command on the server where AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. Note that:- id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
- chip_id: chip ID, which is obtained by running the npu-smi info -m command.
- For the Atlas 350 Accelerator Card: Run the npu-smi info -t board -i id command on the server where AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234.
id indicates the device ID, which is the NPU ID obtained by running the npu-smi info -l command.
Basic functions (operator development, build, and deployment based on the project) are applicable across operator projects created based on the same AI processor series.
- For the following products: Run the npu-smi info command on the server where AI processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.
- -lan: cpp indicates that the operator is developed based on the Ascend C programming framework and using the C/C++ programming language.
- -out: output path, which can be either absolute or relative. The user who runs the tool must have the read and write permissions on the path. If this option is not specified, the outputs are generated to the current path where the command is executed.
- After the command is executed, the operator project directory is generated in the specified directory or default path specified by -out. The project contains the operator implementation template file and compilation script. The following uses the AddCustom operator as an example. The directory structure is as follows:
AddCustom ├── build.sh // Compilation entry script ├── cmake // Directory that stores the scripts used for operator project build and common build files ├── CMakeLists.txt // Build script of the operator project ├── CMakePresets.json // Compilation configuration item ├── framework // Directory of the operator plugin implementation files during AI framework adaptation ├── op_host // Implementation file on the host │ ├── add_custom_tiling.h // Operator tiling definition file │ ├── add_custom.cpp // Content file for operator prototype registration, shape derivation, information library, and tiling implementation │ ├── CMakeLists.txt ├── op_kernel // Implementation file on the kernel │ ├── CMakeLists.txt │ ├── add_custom.cpp // Operator implementation file └── scripts // Directory of scripts used for custom operator project packing
- You simply need to pay attention to the files in bold during subsequent development.
- All files required for implementation on the kernel must be stored in op_kernel. Failure to comply will cause online compilation failure in source release. This is because only the files in op_kernel will be packaged and released during subsequent operator source release.
op_kernel and op_host in the project directory contain the core implementation files of the operator. Operator implementation on the kernel is stored in op_kernel. Code implementation on the host is stored in op_host, including the operator prototype definition and tiling implementation on the host. For details about the operator implementation on the kernel and tiling implementation on the host, see SIMD Operator Implementation. This section focuses on the programming mode and API usage after the CANN framework is connected. You can use the CMakePresets.json file in the project directory to configure project compilation and deploy the project.