Development Through Process Orchestration

The samples described in this section apply to the Atlas inference product and Atlas 200I/500 A2 inference product.

Sample Overview

The following uses Atlas inference products as an example to describe how to develop an inference application using Vision SDK process orchestration. A YOLOv3 model is used as an example to classify images and output the classification result. The used YOLOv3 model is developed based on the TensorFlow framework.

Preparations

  1. Install and deploy Vision SDK, and then perform the quick start sample.
    Table 1 Required dependencies

    Dependency

    Version

    Link

    OS

    For details, see Supported Hardware and Operating Systems.

    -

    System dependency

    -

    Ubuntu or CentOS

    CANN development kit

    8.1.RC1

    Click here to download CANN.

    npu-driver

    Ascend HDK 25.0.RC1

    Click here. In the Select Resource area on the left, filter the required software packages, confirm the version information, and download the software packages.

    For details, see Driver and Firmware Installation and Upgrade Guides of each hardware product.

    npu-firmware

    Ascend HDK 25.0.RC1

  2. Obtain the sample code.

    Click here to obtain the sample code package.

  3. Log in to the development environment where Vision SDK is installed, and upload the sample code package.
  4. Decompress the sample code package and go to the decompressed directory.
    1
    2
    unzip pipelineSample.zip 
    cd pipelineSample
    

    The following is an example of the directory structure of the sample code.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    |-- pipelineSample
    |   |-- data
    |   |   |-- dog1_1024_683.jpg            // Test image
    |   |-- models                        // Directory for storing the model
    |   |   |-- yolov3_tf_bs1_fp16.cfg     // Model configuration file
    |   |   |-- aipp_yolov3_416_416.aippconfig  // YOLOv3.om model AIPP conversion file
    |   |   |-- yolov3.names   // Model output class name file
    |   |-- pipeline                     // Path for storing the pipeline file.
       |   |-- Sample.pipeline        // Pipeline file
    |   |-- src
    |   |   |-- CMakeLists.txt              // CMakeLists file
    |   |   |-- main.cpp                    // Implementation file of the main function, for image classification
    |   |-- README.md
    |   |-- run.sh                   // Script for running the program. It is recommended that you use the dos2unix tool to run the dos2unix run.sh command to format the script before running the program.
    
  5. Prepare the yolov3_tf.pb model for inference by referring to section "Preparing a Model" in README.md (see the decompression directory in 4).
  6. Prepare image data for inference.

    You need to use the obtained image for the test. (Change the name of the obtained image to be the same as that in the sample code, for example, dog1_1024_683.jpg.) The following figure is for display purposes.

    Figure 1 Typical sample image

Pipeline File Orchestration

Pipeline file orchestration is the core task of using Vision SDK to develop applications. An image classification application consists of a series of service processes. The pipeline file is edited first, and then the Vision SDK plugin library is invoked for the inference service. The pipeline file used in this section is orchestrated based on the service process shown in Figure 2.

Figure 2 Service process orchestration

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
{
  "objectdetection": {                // Change "objectdetection" to the name of the current inference process.
    "stream_config": {
      "deviceId": "0"                // "deviceId" indicates the ID of the processor to be used.
    },
    "appsrc0": {                     // "appsrc0" indicates the name of the input element.
      "props": {                     // "props" indicates the element attribute.
        "blocksize": "409600"        // Size of data read by each buffer
      },
      "factory": "appsrc",           // "factory" defines the element type.
      "next": "mxpi_imagedecoder0"   // Enter the connected downstream element. Here, an image decoding element is used.
    },
    "mxpi_imagedecoder0": {          // Name of the image decoding element. The value 0 indicates the ID. If multiple image decoding elements are used in a process, name elements as 0, 1, 2, and so on.
      "props": {
        "handleMethod": "ascend"     // The decoding method is ascend.
      },
      "factory": "mxpi_imagedecoder",    // Use an image decoding plugin.
      "next": "mxpi_imageresize0"    // Enter the connected downstream element. Here, an image resizing element is used.
    },
    "mxpi_imageresize0": {           // Name of the image resizing element
      "props": {
        "handleMethod": "ascend"    // The decoding method is ascend.
        "resizeHeight": "416",       // Height after resizing
        "resizeWidth": "416",        // Width after resizing
        "resizeType": "Resizer_Stretch"// The stretch mode is used for resizing.
      },
      "factory": "mxpi_imageresize",   // Use an image resizing plugin.
      "next": "mxpi_tensorinfer0"     // Enter the connected downstream element. Here, a model inference element is used.
    },
    "mxpi_tensorinfer0": {                                   // Name of the model inference element
      "props": {                                             // "props" indicates the element attribute, which can be used to load files in a specified directory.
        "dataSource": "mxpi_imageresize0",              // Enter the connected upstream element. Here, an image resizing element is used.
        "modelPath": "../models/yolov3_tf_bs1_fp16.om",    // "modelPath" defines the model used for the inference service. Change the file named based on the obtained model.
        "waitingTime": "2000",                               // Waiting time for batch combination allowed by the multi-batch model
        "outputDeviceId": "-1"                               // Copy the memory to the specified location. If this parameter is set to -1, the memory is copied to the host.
      },
      "factory": "mxpi_tensorinfer",                         // Use a model inference plugin.
      "next": "mxpi_objectdetection0"                     // Enter the connected downstream element. Here, a model postprocessing element is used.
    },
    "mxpi_objectdetection0": {                            // Name of the model postprocessing element
      "props": {                                             // "props" indicates the element attribute, which can be used to load files in a specified directory.
        "dataSource": "mxpi_tensorinfer0",                   // Enter the connected upstream element. Here, a model inference element is used.
        "postProcessConfigPath": "../models/yolov3_tf_bs1_fp16.cfg",// "postProcessConfigPath" specifies the postprocessing configuration file of the model.
        "labelPath": "../models/yolov3.names",    // "labelPath" specifies the category name file output by the model.
        "postProcessLibPath": "libyolov3postprocess.so"    // "postProcessLibPath" specifies the dynamic library on which model postprocessing depends.
      },
      "factory": "mxpi_objectpostprocessor",                  // Use a model postprocessing plugin.
      "next": "mxpi_dataserialize0"                          // Enter the connected downstream element. Here, a serialization element is used.
    },
    "mxpi_dataserialize0": {                                 // Name of the serialization element
      "props": {
        "outputDataKeys": "mxpi_objectdetection0"         // "outputDataKeys" specifies the index of the data to be output.
      },
      "factory": "mxpi_dataserialize",                       // Use a serialization plugin.
      "next": "appsink0"                                     // Enter the connected downstream element. Here, an output element is used.
    },
    "appsink0": {                                            // Name of the output element
      "props": {
        "blocksize": "4096000"                               // Size of data read by each buffer
      },
      "factory": "appsink"                                   // Use an output plugin.
    }
  }
}

Comments in the pipeline file are used only for understanding. When compiling the pipeline file, delete the comments. Otherwise, the file fails to be parsed.

The key concepts in the preceding pipeline are as follows:

  • next indicates the connection between elements.
  • appsrc0 is used to send data to a stream, and appsink0 is used to obtain the inference result from the stream.
  • mxpi_objectdetection0 is used to postprocess the output tensor for model inference. In the preceding example, the model postprocessing element processes the one-dimensional tensor output by the upstream model inference element and returns the model detection result (coordinates, class, and confidence score).
  • mxpi_dataserialize0 assembles the inference result into a JSON character string for output.

Code Parsing

The main.cpp file in the pipelineSample/src directory is the application source code.

In this example, the key steps and code are as follows. You cannot directly copy, compile, and run the pipeline file. You need to modify the pipeline file path, input image path, and stream name based on the actual situation. The stream name must match the name of the service inference process in the pipeline file, for example, objectdetection. For details about the complete sample code, see the sample file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
int main(int argc, char* argv[])
 {
    // 1. Parse the pipeline file.
    std::string pipelineConfigPath = "../pipeline/Sample.pipeline";  // Modify the pipeline file path.
    std::string pipelineConfig = ReadPipeline(pipelineConfigPath);
    if (pipelineConfig == "") {
        LogError << "Read pipeline failed.";
        return APP_ERR_COMM_INIT_FAIL;
    }
    // 2. Initialize the stream manager.
    MxStream::MxStreamManager mxStreamManager;
    APP_ERROR ret = mxStreamManager.InitManager();
    if (ret != APP_ERR_OK) {
        LogError << GetErrorInfo(ret) << "Failed to init Stream manager.";
        return ret;
    }
    // 3. Create a stream.
    ret = mxStreamManager.CreateMultipleStreams(pipelineConfig);
    if (ret != APP_ERR_OK) {
        LogError << GetErrorInfo(ret) << "Failed to create Stream.";
        mxStreamManager.DestroyAllStreams();
        return ret;
    }
    // 4. Read the image to be inferred.
    MxStream::MxstDataInput dataBuffer;
    ret = ReadFile("../data/dog1_1024_683.jpg", dataBuffer);    // Change the input image path.
    if (ret != APP_ERR_OK) {
        LogError << GetErrorInfo(ret) << "Failed to read image file.";
        mxStreamManager.DestroyAllStreams();
        return ret;
    }
    std::string streamName = "objectdetection";    // Change the name of the inference process.
    int inPluginId = 0;
    // 5. Send the image to be inferred to the stream.
    ret = mxStreamManager.SendData(streamName, inPluginId, dataBuffer);
    if (ret != APP_ERR_OK) {
        LogError << GetErrorInfo(ret) << "Failed to send data to stream.";
        delete dataBuffer.dataPtr;
        dataBuffer.dataPtr = nullptr;
        mxStreamManager.DestroyAllStreams();
        return ret;
    }
    // 6. Obtain the inference result.
    MxStream::MxstDataOutput* output = mxStreamManager.GetResult(streamName, inPluginId);
    if (output == nullptr) {
        LogError << "Failed to get pipeline output.";
        delete dataBuffer.dataPtr;
        dataBuffer.dataPtr = nullptr;
        mxStreamManager.DestroyAllStreams();
        return ret;
    }
    std::string result = std::string((char *)output->dataPtr, output->dataSize);
    std::cout << "Results:" << result << std::endl;
    // 7. Destroy the stream and resources.
    mxStreamManager.DestroyAllStreams();
    delete dataBuffer.dataPtr;
    dataBuffer.dataPtr = nullptr;
    delete output;
    return 0;
 }

Application Compilation and Running

  1. Log in to the development environment where Vision SDK is installed and go to the pipelineSample/src directory.
  2. Configure environment variables. (The default CANN installation path /usr/local/Ascend/cann and the Vision SDK installation path /home/mxVision-{version} are used as examples.)
    1
    2
    source /usr/local/Ascend/cann/set_env.sh
    source /home/mxVision-{version}/set_env.sh
    
  3. Run the application and execute the compilation script.
    1
    2
    chmod +x run.sh
    ./run.sh
    

    The following information is displayed, in which classId indicates the class ID, className indicates the class name, and confidence indicates the maximum confidence score of the class.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    Results:{
        "MxpiObject":[{"classVec":[{
            "classId":16,
            "className":"dog",
            "confidence":0.994434595,
            "headerVec":[]}],
        "x0":113.476166,
        "x1":882.497559,
        "y0":127.61911,
        "y1":595.543884
        }]
    }