AI CPU Operator Information Library

Configuration Description

Configure the operator information library file to register the operator implementation information on Ascend AI Processor to the operator information library.

The operator information library file is stored in cpukernel/op_info_cfg/aicpu_kernel/xx.ini in the custom operator project directory.

**Table 1** Configurations description of operator information library
Option	Required	Description
[OpType]	Yes	It defines the start of operator information and must be the same as the *OpType* in *REG_OP(OpType)* in Operator Prototype Definition.
opInfo.engine	Yes	Operator engine. Fixed to DNN_VM_AICPU for AI CPU custom operators.
opInfo.flagPartial	Yes	Reserved, fixed to False.
opInfo.computeCost	Yes	Reserved, fixed to 100.
opInfo.flagAsync	Yes	Reserved, fixed to False.
opInfo.opKernelLib	Yes	Kernel library called by the operator. Fixed to CUSTAICPUKernel for AI CPU custom operators.
opInfo.kernelSo	Yes	Name of the dynamic library file (.so) generated after the AI CPU operator implementation is built. The generated .so file is named after AICPU_KERNEL_TARGET configured in the build.sh script obtained in Operator Project Building. For example, if AICPU_KERNEL_TARGET is set to cust_aicpu_kernels_3.3.0, the generated .so file is named libcust_aicpu_kernels_3.3.0.so.
opInfo.functionName	Yes	Name of the kernel function called by the custom operator. If opInfo.flagSupportBlockDim is set to False, block-based parallel execution is not required for the AI CPU operator, and *functionName* must be set to RunCpuKernel. If opInfo.flagSupportBlockDim is set to True, block-based parallel execution is required for the AI CPU operator. In this case, the value of opInfo.functionName must be set to RunCpuKernelWithBlock.
opInfo.workspaceSize	No	Memory size for recording AI CPU operator logs. The value ranges from 0 to 500, in KB. The default value is 2. Set this parameter based on the size of the log content to be recorded. When the size of the log to be recorded exceeds the configured value, the excess logs are not recorded.
opInfo.subTypeOfInferShape	No	AI CPU InferShape mode in the dynamic shape scenario. The values are as follows: 1: The output shape can be directly inferred based on the input shape. 2: Limited output shape can be simply computed based on tensorData. 3: The output shape can be fully computed and the maximum shape of the operator can be inferred. Memory is allocated based on the maximum shape of the operator. 4: The output shape and output range of the operator cannot be inferred from the input shape. In this case, memory needs to be allocated during operator implementation. After the calculation is complete, the output and its shape are returned. Defaults to 1.
opInfo.flagSupportBlockDim	No	BlockDim flag for an AI CPU operator. The values are as follows: True: sets the BlockDim flag, that is, supporting block-based parallel computing. False: does not set the BlockDim flag, that is, block-based parallel computing is not supported. Defaults to False.
opInfo.blockDimByIndex	No	BlockDim partition rule. The values are as follows: -1: BlockDim is partitioned based on the number of elements of the first operator input. An integer within [0, N – 1]. N indicates the dimension count of the shape of the first input. For example, if the value is 0, BlockDim is partitioned based on axis 0 of the first input. If the value is 1, BlockDim is partitioned based on axis 1 of the first input. Defaults to -1. Valid only when opInfo.flagSupportBlockDim is set to True.
input0.name	No	Name of input0, which must be the same as that of input0 under *REG_OP(OpType)* in Operator Prototype Definition. NOTE: The name, type, and format fields of the operator inputs and outputs are used for argument validation at the operator build time. If they are not specified, the arguments are not validated until the operator run time. It is advisable to configure this option.
input0.type	No	Data type supported by input0. If multiple data types are supported, separate them with commas (,). Selected from: DT_UINT8, DT_UINT16, DT_INT8, DT_INT16, DT_INT32, DT_INT64, DT_FLOAT16, DT_FLOAT, DT_DOUBLE, DT_BOOL, and more. Must be consistent with the TensorType list in Operator Prototype Definition. Note: Do not insert a space between data types.
input0.format	No	Supported format. Selected from ND, NHWC, and NCHW. Defaults to ND, indicating that there is no requirement on the format.
......	No	The rest configurations for input1. Configure other inputs (if any, for example, input2, input3, ...) of the operator by referring to input0's configurations.
dynamic_inputX.name	No	Dynamic input configuration. In dynamic_inputX.name, X indicates the sequence number of the operator input. For example, if the first input of the operator is a static input, the input is denoted as input0; if the second input is a dynamic input, this parameter is changed to dynamic_input1.name. Pass the name of the dynamic input for this parameter, which must be the same as the name of the corresponding dynamic input in Operator Prototype Definition. For example, x0, x1, x2, and more are automatically generated based on the actual number of inputs if the input name is x. The sequence number are indexed from 0 in ascending order.
dynamic_inputX.type	No	Supported data type (or data types separated with commas) of the dynamic input. Selected from: DT_UINT8, DT_UINT16, DT_INT8, DT_INT16, DT_INT32, DT_INT64, DT_FLOAT16, DT_FLOAT, DT_DOUBLE, DT_BOOL, and more. Note: X in dynamic_inputX.type indicates the sequence number. Its meaning is the same as X in dynamic_inputX.name.
output0.name	No	Name of output0, which must be the same as that of output0 under *REG_OP(OpType)* in Operator Prototype Definition. If the operator has output0, the name parameter of output0 must be configured.
output0.type	No	Data type supported by output0. If multiple data types are supported, separate them with commas (,). Selected from: DT_UINT8, DT_UINT16, DT_INT8, DT_INT16, DT_INT32, DT_INT64, DT_FLOAT16, DT_FLOAT, DT_DOUBLE, DT_BOOL, and more.
output0.format	No	Supported format. Selected from ND, NHWC, and NCHW. Defaults to ND, indicating that there is no requirement on the format.
......	No	The rest of configurations for output1. Configure other outputs (if any, for example, output2, output3, ...) of the operator by referring to output0's configurations.
dynamic_outputX.name	No	Dynamic output configuration. In dynamic_outputX.name, X indicates the sequence number of the operator output. For example, if the first output of the operator is a static output, the output is denoted as output0; if the second output is a dynamic output, this parameter is changed to dynamic_output1.name. Pass the name of the dynamic output for this parameter, which must be the same as the name of the corresponding dynamic output in Operator Prototype Definition. For example, y0, y1, y2, and more are automatically generated based on the actual number of outputs if the output name is y. The sequence number are indexed from 0 in ascending order.
dynamic_outputX.type	No	Supported data type (or data types separated with commas) of the dynamic output. Selected from: DT_UINT8, DT_UINT16, DT_INT8, DT_INT16, DT_INT32, DT_INT64, DT_FLOAT16, DT_FLOAT, DT_DOUBLE, DT_BOOL, and more. Must be consistent with the TensorType list in Operator Prototype Definition. Note: X in dynamic_outputX.type indicates the sequence number. Its meaning is the same as X in dynamic_outputX.name.

A configuration example is provided as follows:

[Less]
opInfo.engine=DNN_VM_AICPU
opInfo.flagPartial=False
opInfo.computeCost=100
opInfo.flagAsync=False
opInfo.opKernelLib=CUSTAICPUKernel
opInfo.kernelSo=libcust_aicpu_kernels.so
opInfo.functionName=RunCpuKernel
opInfo.workspaceSize=2
input0.name=x1
input0.type=DT_DOUBLE,DT_FLOAT,DT_FLOAT16,DT_INT16,DT_INT32,DT_INT64,DT_INT8,DT_UINT16,DT_UINT32,DT_UINT64,DT_UINT8
input1.name=x2
input1.type=DT_DOUBLE,DT_FLOAT,DT_FLOAT16,DT_INT16,DT_INT32,DT_INT64,DT_INT8,DT_UINT16,DT_UINT32,DT_UINT64,DT_UINT8
output0.name=y
output0.type=DT_BOOL

After the custom operator information library file is built and deployed, the operator definition information is stored in <path>/<vendor_name>/op_impl/cpu/config/cust_aicpu_kernel.json, where <path> indicates the OPP installation directory (see OPP Deployment) and <vendor_name> indicates the name of the vendor that provides the custom operator. For details about the definition of a vendor name, see Operator Project Building.

Parent topic: Operator Information Library Definition