Call Process

Workflow

Figure 1 shows the workflow.

1. Install CANN in the development environment, obtain ATC from the installation path, and set environment variables. For details, see Environment Setup.
2. Prepare a model or a single-operator description file to be converted and upload it to the development environment. For details about the configuration of the single-operator description file, see Generating a Single-Operator Model.
3. Use ATC to convert the model. For details about the parameters used during model conversion, see Command Line Options.

Interaction in Model Conversion

The following uses the conversion from the open-source model into the OM offline model as an example to describe the interaction between ATC and other related modules in model conversion.

In a network model, operators are classified into AI Core operators and AI CPU operators based on the compute units used. AI Core operators are executed on the AI Core compute units of the AI processor, while AI CPU operators are executed on the AI CPU compute units.

Although the interaction in model conversion of both AI Core and AI CPU operators involves nodes such as graph preparation, partitioning, optimization, and build, the internal interaction modules of the two types of operators are different because their compute units are different. For details, see the following figure.

For details about operator types and basic concepts, see Ascend C Operator Development. If your model contains custom operators, develop and deploy custom operators by referring to the preceding manual. During model conversion, the custom OPP will be preferentially looked up for operators in the user model, rather than the built-in OPP.

During model conversion, if model build fails due to data type incompatibility of an AI CPU operator, you can enable the Auto Cast feature (the Cast operator is automatically inserted to cast the data type to a supported one). For details, see How Do I Enable Auto Cast for AI CPU Operators?.

• Interaction in model conversion using AI Core operators
  Figure 2 Interaction in model conversion using AI Core operators
  
  1. The Parser is called to parse the original model into the format compatible with CANN.
  2. Graph preparation: The source graph is optimized and shape inference (including setting operator output shape and dtype) is performed using InferShape in this phase.
  3. Graph partitioning: The graph engine (GE) partitions the graph into subgraphs.
  4. Graph optimization: GE optimizes the partitioned subgraphs. During optimization, GE precompiles the AI Core operators based on the data flow of subgraphs, performs Unified Buffer (UB) fusion on these operators on subgraphs based on the fusion patterns, locates the operator implementation file based on the operator information library, builds the operator implementation file into operator kernel files (.o and .json files), and returns the optimized subgraphs to GE. The Atlas 350 Accelerator Card does not support UB fusion.

     GE merges the optimized subgraphs into a graph and further optimizes the graph.

  5. Graph build: GE builds the graph, including memory and stream resource allocation. After the graph build is complete, an OM offline model file that adapts to the AI processor is generated.
• Interaction in model conversion using AI CPU operators
  Figure 3 Interaction in model conversion using AI CPU operators
  
  1. The Parser is called to parse the original model into the format compatible with CANN.
  2. Graph preparation: The basic operator parameter verification is performed and shape inference (including setting operator output shape and dtype) is performed using InferShape in this phase.

     In addition, GE delivers the entire graph to AI CPU Engine. AI CPU Engine reads the operator information library, looks up an appropriate format for the operator, and returns the format to GE.

  3. Graph partitioning: GE partitions the graph into multiple subgraphs.
  4. Graph optimization: GE sends the subgraphs to AI CPU Engine. AI CPU Engine optimizes the subgraphs and returns the optimized subgraphs to GE.

     GE merges the optimized subgraphs into a graph and further optimizes the graph.

  5. Graph build: GE builds the graph (including memory and stream allocation) and sends a genTask request to AI CPU Engine. Then, AI CPU Engine returns the task information of the operator to GE. After the graph is built, an OM offline model file that adapts to the AI processor is generated.