Graph Build and Execution

Environment Requirements

• You have installed the CANN driver and software and set basic environment variables by referring to Environment Setup.

  After the CANN software is installed, log in to the environment as the CANN operating user and run the source ${INSTALL_DIR}/set_env.sh command to set environment variables. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

• You have developed and deployed operators by referring to Project-based Operator Development.

Preparing a Verification Code Project

├── aclop_invocation
│   ├── add_custom.json                   // Operator description file, used to construct a single-operator model file
│   ├── CMakeLists.txt
│   ├── main.cpp                          // File used to compile a single-operator into an .om file and load the .om file for execution

Generating a Single-Operator Offline Model File

1. Construct the static-shape single-operator description file add_custom_static_shape.json to describe the input, output, and attributes of the operator.
   The following is an example of the description file of the AddCustom static-shape operator:

   [
       {
           "op": "AddCustom",
           "input_desc": [
               {
                   "name": "x",
                   "param_type": "required",
                   "format": "ND",
                   "shape": [8, 2048],
                   "type": "float16"
               },
               {
                   "name": "y",
                   "param_type": "required",
                   "format":"ND",
                   "shape": [8, 2048],
                   "type": "float16"
               }
           ],
           "output_desc": [
               {
                   "name": "z",
                   "param_type": "required",
                   "format":  "ND",
                   "shape": [8, 2048],
                   "type": "float16"
               }
           ]
       }
   ]

2. Use the ATC tool to compile the operator description file into a single-operator model file (*.om file).

   The following is a command example of the ATC tool:

   atc --singleop=../add_custom_static_shape.json --output=. --soc_version=<soc_version>

   The key command-line options are described as follows. For details, see ATC. :

   • --singleop: path of the single-operator description file (JSON format).
   • --output: directory for storing OM files.
   • --soc_version: AI Processor version. Replace it with the actual version.
     

     The AI processor model can be obtained in the following ways:

     • For the following products: Run the npu-smi info command on the server where AI processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.

       Atlas A2 training product/Atlas A2 inference product

       Atlas 200I/500 A2 inference product

       Atlas inference product

       Atlas training product

     • For the Atlas A3 training product/Atlas A3 inference product: Run the npu-smi info -t board -i id -c chip_id command on the server where AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. Note that:
       • id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
       • chip_id: chip ID, which is obtained by running the npu-smi info -m command.
     • For the Atlas 350 Accelerator Card: Run the npu-smi info -t board -i id command on the server where AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234.

       id indicates the device ID, which is the NPU ID obtained by running the npu-smi info -l command.

   After the preceding command is executed, an offline model file with the suffix *.om is generated in the path specified by the --output option.

Compiling Verification Code

Compile the code logic for loading and executing a single operator by referring to the following example.

The following is a code snippet of key steps only, which is not ready to be built or run. After APIs are called, you need to add exception handling branches and record error logs and info logs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

// 1. Perform initialization.
CHECK_ACL(aclInit(nullptr));

// 2. Allocate runtime resources.
const int32_t deviceId = 0;
CHECK_ACL(aclrtSetDevice(deviceId));

// 3. Load the single-operator model file (*.om file).
CHECK_ACL(aclopSetModelDir("."));

// 4. Set the operator input, allocate memory, read the input data, and save them to the allocated memory.
// ......

// 5. Create a stream.
aclrtStream stream = nullptr;
aclrtCreateStream(&stream);

// 6. Execute the single-operator.
// opType indicates the operator type name, for example, AddCustom.
// inputDesc.size() indicates the number of operator inputs. For example, the AddCustom operator has two inputs.
// inputDesc.data() indicates the array of the operator input tensor descriptions, describing the format, shape, and data type of each input.
// inputBuffers.data() indicates the input tensor data of the operator.
// outputDesc.size() indicates the number of operator outputs. For example, the AddCustom operator has one output.
// outputDesc.data() indicates the array of the operator output tensor descriptions, describing the format, shape, and data type of each output.
// outputBuffers.data() indicates the output tensor data of the operator.
// opAttr indicates the operator attributes. If the operator does not have attributes, call aclopCreateAttr to create data of the aclopAttr type.
// stream is used to maintain the execution sequence of some asynchronous operations.

CHECK_ACL(aclopExecuteV2(opType, inputDesc.size(), inputDesc.data(), inputBuffers.data(),
                             outputDesc.size(), outputDesc.data(), outputBuffers.data(), opAttr, stream));


// 7. Block the app until all tasks in the specified stream are complete.
aclrtSynchronizeStream(stream);

// 8. Process the output data after the operator is executed, for example, printing data to the screen or writing the data to a file. You can implement this function as required.
// ......

// 9. Destroy the stream.
aclrtDestroyStream(stream);

// 10. Destroy runtime allocations.
aclRet = aclrtResetDevice(deviceId);
aclRet = aclFinalize();

// ....

Running and Verification

1. In the development environment, set environment variables and configure the paths of the header files and library files on which the build of single-operator verification program depends. The following is an example of setting environment variables. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann. {arch-os} indicates the architecture and OS of the operating environment. arch indicates the OS architecture, and os indicates the operating system, for example, x86_64-linux.

   export DDK_PATH=${INSTALL_DIR}
   export NPU_HOST_LIB=${INSTALL_DIR}/{arch-os}/devlib

2. Build the sample project to generate an executable file for single-operator verification.
   1. Go to the directory of the sample project and run the following command in this directory to create a directory (for example, build) for storing the generated executable file.

      mkdir -p build

   2. Go to the build directory and run the CMake compile command to generate build files.

      Command example:

      cd build

      cmake ../src -DCMAKE_SKIP_RPATH=TRUE

   3. Run the following command to generate an executable file:

      make

      The executable file execute_add_op is generated in the output directory of the project.

3. Execute the single-operator.
   1. Copy execute_add_op in the output directory of the sample project in the development environment to any directory in the operating environment as the running user (for example, HwHiAiUser).

      Note: If your development environment is the operating environment, skip this step.

   2. Execute the execute_add_op file in the operating environment to verify the single-operator model file.

      chmod +x execute_add_op

      ./execute_add_op

      If "test pass" is displayed, the execution is successful.