Process Orchestration

Service Process

By using the pipeline configuration file, the stream manager can identify the elements to be constructed and the connection between the elements, and start the service process. The stream manager provides APIs for external systems to send data to and obtain results from the stream, helping implement service interconnection.

A plugin is the basic module in the service process. A stream is constructed by concatenating elements. The buffer is used to mount the video and image data before and after decoding, which is the data structure transferred between elements. In addition, the buffer allows users to mount metadata to store structured data (such as the object detection result) or process data (such as the resized image).

Data Flow

To help you understand the data structure transfer process between plugins, the following procedure uses an image as the input and describes the data transfer process in detail based on the code.

1. Create a variable MxstDataInput for storing the input image and use the ReadFile function to read the image data to the variable dataBuffer.

   1 2	MxStream::MxstDataInput dataBuffer; APP_ERROR ret = ReadFile("./test.jpg", dataBuffer);

2. Create a stream management object MxStreamManager, initialize the object, and load the pipeline file.

   1 2 3 4

   MxStreamManager *mxStreamManager = new MxStream::MxStreamManager(); ret = mxStreamManager->InitManager(); string streamsConfig = "detection.pipeline"; // The detection.pipeline file can contain one or more service streams. APP_ERROR ret = mxStreamManager->CreateMultipleStreams(streamsConfig); // The method for processing multiple service streams is used.

3. Use the SendData function to pass the input data to the image obtaining module.

   1	ret = mxStreamManager->SendData(streamName, inPluginId, dataBuffer); // streamName is the name of the service stream in the pipeline file. inPluginId indicates the input port number, corresponding to the input element number.

4. Process data in the Process function of each plugin.
   1. Receive the buffer data of the previous plugin and the metadata mounted to the buffer.

      1 2 3

      MxpiFrame inputMxpiFrame = GetHostDataInfo(*mxpiBuffer); // If the data is on the device, call the GetDeviceDataInfo function. MxpiMetadataManager mxpiMetadataManager(*mxpiBuffer); // Create a metadata manager object. shared_ptr<void> metadata = mxpiMetadataManager.GetMetadata("keyName"); // Obtain the metadata mounted to the buffer based on the key name. keyName is the name of the previous plugin.

   2. Process the service logic.

      1 2

      DataProcess(inputMxpiFrame, metadata, outputMetadata); // Data processing functions implemented in the plugin, such as decoding, resizing, and inference. mxpiMetadataManager.AddMetadata("keyName", outputMetadata); // Mount outputMetadata to the buffer. keyName is usually the name of the current plugin.

   3. Send buffer data to the next plugin.

      1	SendData(inPlugin, *mxpiBuffer); // Call the SendData method of MxPluginBase to send data to the next plugin.

5. Use the GetResult function to obtain the output result of the last plugin.

   1	MxStream::MxstDataOutput* output = mxStreamManager->GetResult(streamName, outPluginId); // outPluginId indicates the output port number, corresponding to the output element number.

Introduction to the Pipeline Configuration File

Figure 3 and Figure 4 are pipeline configuration file samples for an inference service stream, including a service stream name, stream configuration, an element name, a plugin name, element properties, and a downstream element name.

The configuration file is in JSON format. You must specify the service stream name, element name, and plugin name, and add element properties and the downstream element name as required. Otherwise, the stream will fail to be created and errors will occur. For details, see Table 2.

Common attributes can be directly configured under props. For details, see Table 3.

Process Orchestration for Multi-input/output Plugins

If an element contains multiple output ports, use square brackets ([]) to enclose multiple elements specified by next and commas (,) to separate the elements, as shown in Figure 5.

Setting deviceId in Batches

Add deviceId (default value: 0) corresponding to the stream_config field to the pipeline configuration file. In this way, you can set device IDs in batches, as shown in Figure 6.

Code Process Orchestration

Code process orchestration refers to creating streams by using a composition method similar to a deep learning framework to avoid writing complex pipeline files. In this way, the process orchestration can be quickly and efficiently implemented. Table 4 lists the key functions. For details about the compilation example, see Figure 7.