Process Orchestration

Figure 1 shows the basic units related to the SDK service process. For details about the basic units, see Table 1.

Basic SDK Concepts

By using the pipeline configuration file, the stream manager can identify the elements to be constructed and the connection between the elements, and start the service process. The stream manager provides APIs for external systems to send data to and obtain results from the stream, helping implement service interconnection.

A plugin is the basic module in the service process. A stream is constructed by concatenating elements. The buffer is used to mount the video and image data before and after decoding, which is the data structure transferred between elements. In addition, the buffer allows users to mount metadata to store structured data (such as the target detection result) or process data (such as the resized image).

Figure 1 Basic units related to the SDK service process

**Table 1** Basic concepts
Name	Class Name	Description	Description
Stream	MxStream	Service stream	A complete inference service stream, which is concatenated by plugins.
Stream manager	MxStreamManager	Service stream management module	Builds and destroys streams, and provides APIs for sending data and obtaining results.
Plugin	MxPlugin	Function plugin	A functional module in a service stream, which is the basic unit of the service stream.
Element	None	Function element	Object generated after the .so plugin is instantiated. The same .so plugin can instantiate multiple objects, as shown in the two image resizing elements in Figure 1.
Plugin Buffer	MxpiBuffer	Plugin buffer	Unstructured data transmitted between plugins, such as video and image data before and after decoding.
Plugin Metadata	MxpiMetadata	Plugin metadata	Structured data generated by the plugin, such as classification information and target information. Metadata is attached to the plugin buffer for transmission.

Introduction to the Pipeline Configuration File

Figure 2 is a pipeline configuration file sample for an inference service stream, including a service stream name, stream configuration, an element name, a plugin name, element properties, and a downstream element name.

The configuration file is in JSON format. You must specify the service stream name, element name, and plugin name, and add element properties and the downstream element name as required. For details, see Table 2.

Figure 2 Pipeline configuration file sample

Figure 3 Pipeline configuration file with nextMeta

**Table 2** Pipeline configuration file description
Name	Mandatory or Not	Description
Service stream name (stream)	Mandatory	Specifies the stream entity to be operated. For example, when the stream manager sends data, the stream name needs to be specified.
Stream configuration (stream_config)	Mandatory	Sets the configuration parameters of the stream. For example, Device ID specifies the processor on which the stream runs.
Element name (element)	Mandatory	Constructs a plugin entity in the stream, that is, a plugin object. This item is used together with the downstream element name to define the connection between elements. You are advised to name the item in the format of plugin base class name + serial number and ensure that the name is unique in the stream.
Plugin name (factory)	Mandatory	Used to build elements. Obtain the value from Existing Plugin Introduction.
Element properties (props)	Optional	Used to modify configuration parameters based on service features. For example, in the image resizing plugin, the properties specify the width and height of the output image. Each property has a default value. For details, see Existing Plugin Introduction.
Downstream element name (next)	Mandatory (except the last element in the stream)	Used to determine the functions of the downstream element based on specific services. The data generated by this plugin will be transmitted to the downstream element. If there are multiple downstream plugins, separate them with commas (,). The port number is after the colon (:), and is enclosed in square brackets ([]).
Name of the downstream metadata receiving element (NextMeta)	Optional	Used to set the plugin with multiple input ports to determine functions of the downstream metadata receiving element. The metadata generated by the plugin is transferred to the downstream element. If there are multiple downstream plugins, separate them with commas (,) and enclose them in square brackets ([]). You can add a port number after the colon (:) to specify the input sequence of the same downstream element. If no port number is added, the default input sequence is used.

**Table 3** Parameters for configuring common plugin properties
Name	Mandatory or Not	Description
dataSource	Optional (You are advised to use the nextMeta property.)	Indicates the data processed by the process function of the plugin. It is an upstream SDK plugin, and the native GStreamer plugin is not supported. The value is of the string type, and defaults to auto. This value is specified by nextMeta of one or more upstream plugins. If an upstream plugin does not add nextMeta to the plugin, the plugin uses dataSource. Otherwise, the plugin uses nextMeta.
status	Optional	Specifies whether the plugin is executed synchronously or asynchronously. The value is of the int type and can be 0 or 1. The value 0 (default) indicates asynchronous execution, and the value 1 indicates synchronous execution. Exercise caution when setting this parameter.
deviceId	Optional	Specifies the ID of the processor on the device where the plugin runs. The value is of the int type. The default value is 0. Set this parameter as required.

Common properties can be directly configured under props.

Figure 4 Configuring common properties

The property value is a string regardless of whether the data type of the property is string. For example, "resizeWidth": "2048" is used instead of "resizeWidth": 2048.
Currently, a single stream can run on only one Ascend AI Processor (that is, one device). Specify deviceId by using Stream_config.

Process Orchestration for Multi-input/output Elements

If an element contains multiple input ports, add :index (index indicates the port number) to the end of the element name after next to specify the port, as shown in Figure 5. For example, mxpi_parallel2serial0:0 indicates that the output port of the element is connected to port 0 of the mxpi_parallel2serial0 element.

Figure 5 Example of multi-input element process orchestration

If an element contains multiple output ports, use square brackets ([]) to enclose multiple elements specified by next and commas (,) to separate the elements, as shown in Figure 6.

Figure 6 Example of multi-output element process orchestration

The element serial number specified in [] corresponds to the output port of the current element. That is, the first element in [] is associated with port 0 of the current element, the second element is associated with port 1, and so on.

Parent topic: Basic Development