Documentation

Sample Code for Executing a Static-Shape Operator

This section describes the key APIs and sample code for calling the static-shape operator in single-operator model execution mode.

Prerequisite

Before calling the AscendCL API to execute the Static-Shape operator, you need to compile the operator in advance. The following explains how to use the ATC tool to compile the model file of the Add operator:

  1. Construct the description file (*.json file) of the Add operator, which describes the input and output tensors and operator attributes.
    Example description file of the Add operator:
    [
  {
    "op": "Add",
    "input_desc": [
      {
        "format": "ND",
        "shape": [8, 16],
        "type": "int32"
      },
      {
        "format": "ND",
        "shape": [8, 16],
        "type": "int32"
      }
    ],
    "output_desc": [
      {
        "format": "ND",
        "shape": [8, 16],
        "type": "int32"
      }
    ]
  }
]
  2. Use the ATC tool to compile the operator description file into a single-operator model file (*.om file), and then call the AscendCL APIs to load the OM model file and execute the operator.

    The following is a command example of the ATC tool:

    atc --singleop=$HOME/singleop/add.json --output=$HOME/singleop/out/op_model --soc_version=<soc_version>

    The key parameters are described as follows. For details about the parameters, see ATC Instructions. :

    • --singleop: path of the single-operator description file (JSON format).
    • --output: directory for storing the single-operator model file.
    • --soc_version: version of Ascend AI Processor.

Sample Code

The following is only a code snippet of key steps for single-operator loading and execution, which is not ready to be built or run. Following the API calls, add exception handling branches and specify log printing of error and information levels. The complete code is available in the acl_execute_add sample.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
//1. Initialize AscendCL.
aclRet = aclInit(nullptr);

//2. Allocate runtime resources. (The default context and stream are used. When the default stream is used as the argument of another API, a null pointer can be passed.)
aclRet = aclrtSetDevice(0);
//Obtain the run mode of the software stack. Different run modes lead to different API call sequences (for example, whether data transfer is required).
aclrtRunMode runMode;
bool g_isDevice = false;
aclError aclRet = aclrtGetRunMode(&runMode);
g_isDevice = (runMode == ACL_DEVICE);

//3. Load the single-operator model file (.om file).
//This directory is relative to the directory of the executable file. For example, if the executable file is stored in the out directory, the directory is out/op_models.
aclRet = aclopSetModelDir("op_models");

//4. Execute the operator.
//opType indicates the operator type name, for example, Add.
//numInputs indicates the number of operator inputs. For example, the Add operator has two inputs.
//inputDesc indicates the array of the operator input tensor descriptions, describing the format, shape, and data type of each input.
//inputs indicates the input tensor data of the operator.
//numOutputs indicates the number of operator outputs. For example, the Add operator has one output.
//outputDesc indicates the array of the operator output tensor descriptions, describing the format, shape, and data type of each output.
//outputs indicates the output tensor data of the operator.
//attr indicates the operator attributes. If the operator does not have attributes, call aclopCreateAttr to create data of the aclopAttr type.
//stream is used to maintain the execution sequence of some asynchronous operations.

aclopExecuteV2(opType, numInputs, inputDesc, inputs,                 
               numOutputs, outputDesc, outputs, attr, nullptr);

//Process the output data after the operator is executed, for example, printing the data to the screen or writing the data to a file. You can implement this function as required.
// ......

//Block the app until all tasks in the specified stream are complete.
aclrtSynchronizeStream(nullptr);

//5. Release runtime resources. (By default, the context and stream resources are automatically released using the aclrtResetDevice call.)
aclRet = aclrtResetDevice(0);

//6. Deinitialize AscendCL.
aclRet = aclFinalize();

// ....