Overview

Application Scenarios

If your AI app involves not only model inference but also mathematical operations, such as Basic Linear Algebra Subprogram (BLAS) and data type conversion, Ascend CANN can also play its role to provide related computing capabilities.

Ascend CANN supports single-operator calling, a lightweight mode allowing you to load and execute single-operators directly using the AscendCL APIs, which requires no model building and training.

In addition, you can verify the functions of a custom operator via single-operator calling.

Difference Between Single-Operator Calling and Model Inference

The following flowchart shows the basic common points and differences between single-operator calling and model inference.

• Common points:
  • The following steps are required: AscendCL initialization and deinitialization, and runtime resource allocation and deallocation.
  • Loading and execution are required using different AscendCL APIs for single-operator calling and model inference.
• Differences:
  • Model inference involves model unloading, but single-operator calling does not.

Figure 1 Workflow comparison

Development Workflow

Figure 2 Development workflow

1. Set up the environment.

   For details, see Development and Operating Environment Setup.

2. Create code directories.

   Create a directory to store code files, build scripts, test images, and model files.

   The following is an example:

   ├App name
   ├ ─ ─ op_model // Operator description files for compilation
   │   ├── xxx.json               
   
   ├── data
   │ ├── xxxxxx // Test data
   
   ├── inc                   // Header files that declare functions
   │   ├── xxx.h               
   
   ├── out                   // Output files
   
   ├── src   
   │   ├── xxx.json         // Configuration files for system initialization
   │   ├── CMakeLists.txt   // Build scripts
   │   ├── xxx.cpp          // Implementation files

3. (Optional) Build the operator.

   For operator calling based on Single-Operator API Execution, you do not need to compile the operator.

   For operator calling based on Single-Operator Model Execution, you need to compile the operator in either of the following ways:

   • Generate operator .om files with ATC.

     Construct single-operator description files in *.json format (describing the inputs, outputs, and attributes of the operators), use ATC to compile the single-operator description files into .om files, and call the AscendCL APIs to load the .om files and execute the operators.

     For details about how to use ATC, see ATC Instructions.

   • Call the AscendCL APIs to compile operators.

     Call the AscendCL APIs directly to compile and execute the operators.

   

   For details about single-operator model execution and single-operator API execution and API calling, see Single-Operator Call Sequence.

4. Develop an app.

   For details about the required header files and library files, see Dependent Header Files and Library Files.

   For details about the single-operator call sequence, see Single-Operator Call Sequence and related sample code.

5. Build and run the app. For details, see App Build and Run.