Documentation

Integrated Inference Tool

Overview

To help users quickly call MindStudio functions during model inference, an integrated inference tool AMIT is provided to allow accesses to a range of model analysis, model conversion, offline model dump, accuracy comparison, and performance analysis tools through a single, unified portal, thus enhancing the development efficiency. The AMIT GUI also provides step-by-step guidance to facilitate users' understanding of the inference development process and make it easier to use.

Figure 1 AMIT tool
  1. Model analysis: Analyzes operators that are not supported by a model and allows users to query information about operators supported by the current CANN version and models supported by ModelZoo.
  2. Model conversion: Provides the ATC tool to convert a model into an offline OM model.
  3. Offline model dump: Generates dump data based on the generated offline OM model.
  4. Accuracy comparison: Compares the accuracy difference of computation results of Huawei proprietary operators and third-party equivalents based on the dump data of the offline OM model.
  5. Performance analysis: Uses the Profiling tool to collect and analyze profile data of an application developed using the generated offline OM model.

Procedure

Perform the following steps to learn the operation process of AMIT and quickly get started with inference jobs.

  1. Choose Ascend > AMIT from the menu bar to start the integrated inference tool.
  2. Access the AMIT process page, and click Model Analyse to start a model analysis task, as shown in Figure 2.
    Figure 2 Model analysis
    1. Analyze operators that are not supported by a model.
      1. In Model File, select the model to be analyzed and click to view the model's original network structure. Operators that are not supported by the current CANN version are marked in red, as shown in Figure 3. For more operations, see Description of the Model Visualization GUI.
        Figure 3 Model visualization
      2. Click analyse, and information about operators that are not supported by CANN is displayed in the output window. If no output is generated, all operators in the model are supported, allowing conversion and subsequent development tasks to be performed.
    2. Select a supported information field from the Model Search by drop-down list or enter a keyword in the text box to quickly search for the model. The query result is displayed on the current page. Table 1 shows the parameters.
      Figure 4 Quick model query
      Table 1 Parameters

      Parameter

      Description

      Model Name

      Model name.

      Application Area

      Application field, for example, Image Classification and Object Detection.

      Update Time

      Update time.

      Precision

      Model precision, for example, FP16 and INT8.

      Model Type

      Model type, for example, Offline Model.

      Framework

      Framework, for example, PyTorch and TensorFlow.

      Processor Type

      Processor type, for example, Atlas 200/300/500 inference products or Ascend 910.

      Version

      Model version information.

      Download Link

      Download link. You can copy the link to a browser and open it to obtain the corresponding model file.

      Model Format

      Model format, for example, OM or ONNX.
    3. Enter the operator type name or hardware model in Operator Search by Name to quickly filter the operators to be queried. You can select an operator type to see more details on the right pane.
      Figure 5 Operator details
      • Operator Type
      • Hardware Type: hardware supported by the operator
  3. On the AMIT process page, click Model Converter to enter the parameter configuration page for model conversion. For details about how to use the tool, see Model Conversion. The generated offline OM model can be used for offline model dump and performance analysis tasks.
  4. On the AMIT process page, click Om Dump Configure to enter the model selection page for offline model dump. For details about how to use the tool, see Preparing Dump Data of an Offline Model. The obtained offline model dump data can be used for accuracy comparison tasks.
  5. On the AMIT process page, click Model Accuracy Analyse to enter the parameter configuration page for accuracy comparison. For details about how to use the tool, see Accuracy Comparison.
  6. On the AMIT process page, click System Profiler to enter the parameter configuration page for performance analysis. For details about how to use the tool, see Performance Analysis.