Overview

Application Scenarios

If your AI app performs not only model inference but also mathematical operations, such as Basic Linear Algebra Subprogram (BLAS) and data type conversion, CANN can also play its role to provide related computing capabilities.

CANN supports single-operator calling, a lightweight mode allowing you to load and execute single-operators directly using acl APIs, which requires no model building and training.

You can also use single-operator calling to verify the functions of a custom operator.

Difference Between Single-Operator Calling and Model Inference

The following flowchart shows the basic common points and difference between single-operator calling and model inference.

• Common points: Initialization and deinitialization, as well as runtime resource allocation and deallocation are required.
• Difference: Different acl APIs are required for loading and execution of single-operator calling and model inference.

Development Workflow

1. Prepare the environment.

   For details, see Environment Setup.

2. Create code directories.

   Create directories to store code files, build scripts, test images, model files, and more.

   The following is an example for reference:

   ├App name
   ├ ─ ─ op_model // Operator description files for compilation
   │   ├── xxx.json               
   
   ├── data
   │ ├── xxxxxx // Test data
   
   ├── inc                   // Header files that declare functions
   │   ├── xxx.h               
   
   ├── out                   // Output files
   
   ├── src   
   │   ├── xxx.json         // Configuration files for system initialization
   │   ├── CMakeLists.txt   // Build scripts
   │   ├── xxx.cpp          // Implementation files

3. (Optional) Compile the operator.

   For operator calling based on Single-Operator API Execution, you do not need to compile the operator.

   For operator calling based on Single-Operator Model Execution, you need to compile the operator in either of the following ways:

   • Generate operator .om files with ATC.

     Construct single-operator description files in *.json format (describing the inputs, outputs, and attributes of the operators), use ATC to compile the single-operator description files into .om files, and call acl APIs to load the .om files and execute the operators.

     For details about how to use ATC, see ATC Instructions.

   • You can also call acl APIs to compile and execute the operators.
   

   For details about single-operator model execution and single-operator API execution and API calling, see Single-Operator Call Sequence.

4. Develop an app.

   For details about the required header files and library files, see Dependent Header Files and Library Files.

   For details about the single-operator call sequence, see Single-Operator Call Sequence and related sample code.

5. Build and run the app. For details, see App Build and Run.