Visualizing Process Orchestration

The minimum granularity for MindX SDKs to implement functions is a plugin. Each plugin implements a specific function, such as image decoding and scaling. You can orchestrate these plugins in a proper sequence to implement the corresponding functionality. Visualized process orchestration is to develop data stream diagrams in a visualized manner and generate pipeline files for the application framework.

1. On the project creation page, perform the following operations to start process orchestration:
   • On the top menu bar, choose Ascend > MindX SDK Pipeline. The pipeline drawing page is displayed, as shown in Figure 1.
   • On the top menu bar, choose File > Open... to open the pipeline file you drew, as shown in Figure 2. (If the pipeline file you drew has been stored in the project directory, you can also double-click the file in the project directory to open it.)

   The drawing page consists of the plugin library on the left, editing area in the center, and plugin attribute display area on the right. See Figure 2 for the plugin library on the left and editing area in the center. For details, see Table 1.

   Table 2 describes the buttons in the lower right corner of the window.

   Figure 1 Pipeline drawing page
   

   Figure 2 Drawing a pipeline file
   

   Table 1 Introduction to the drawing page

   Area	Description
   Plugin library	The plugin library is obtained by scanning the installed MindX SDK. The AllPluginsInfo.json file is generated in the project directory and MindStudio identifies plugins based on this file to ensure that the generated pipeline can find the corresponding .so files. You can expand the plugin library directory and drag the required plugins to the editing area. Alternatively, you can search for the plugins by keyword in the search box and drag the plugins to the editing area. The SDK developer toolkit has a built-in plugin library. After the toolkit is installed and the drawing page is displayed, the plugin library is automatically loaded on the left. For details about the built-in plugin library, see "Plugin Reference" in the mxVision User Guide. If the built-in plugin library cannot meet the development requirements, you can develop plugins and import your plugin library using the Plugin Manager. If the target directory does not exist, create it. For details about how to develop a plugin, see Developing a Plugin and "Development Through Process Orchestration" in the mxVision User Guide. If a plugin is changed during project development, you can click in the plugin library to scan and refresh the plugin.
   Editing area	You can drag a plugin from the plugin library to the editing area. Two plugins can be connected to form a data stream. The one-to-one, one-to-many, and many-to-one connection relationships are supported. The configurable attributes of each plugin can be scanned. You can click a plugin in the editing area and customize the configurations (such as the path parameters in the plugin attributes) in the plugin attribute display area. NOTE: In the remote scenario, after selecting the input paths of the model, label, and configuration file for the mxpi_modelinfer plugin, manually change the paths to the absolute paths at the remote end by referring to the procedures performed on Linux. For example, change the model file path to /home/workplace/*.om and ensure that the corresponding file exists in the selected path. The Name field indicates the custom component name. For details, see the mxVision User Guide. As shown in Figure 2, plugins in purple are input plugins, plugins in red are output plugins, and plugins in gray are plugins that are not defined in the current plugin library. Other plugins are in blue. When you hover your cursor over a plugin, the function description of the plugin is displayed. You can press Del on the keyboard to delete the selected plugin or connection. When a plugin or connection is selected, you can scale the plugin block (maximum size: twice the width and height of the plugin block). Open a pipeline file with content. The Pipeline Stream Editor and Text options are displayed in the lower left corner of the editing area. The graphical display (default) and code display modes are supported. You are advised to use the process orchestration visualization function in graphical display mode.
   Plugin attribute display area	Click a plugin to view its attributes. Plugin attributes are displayed in key-value or subkey-subvalue mode. In subkey-subvalue mode, click to delete a key-value pair or click to add a key-value pair.

   Table 2 GUI functions

   Function	Description
   Plugin Manager	Manages custom plugins. With this function, you can add a directory for custom plugins and import a custom plugin library. (The permission on the directory for custom plugins must be 440. Confirm this before importing the plugins.) Click + Add Plugin. In the displayed dialog box, select the directory where the .so file of the custom plugin is located and click OK. After configuring the plugin directory, click Save. Open the pipeline, click in the plugin library, and open the pipeline again as prompted. You can view that the custom plugins have been loaded in the plugin library. When using Plugin Manager to import a custom plugin, pay attention to its dependencies. If dependencies of the plugin are not installed, the plugin may not be displayed in the plugin library.
   Format	Resolves plugin overlapping or formats the plugin process drawn by a user.
   New	Creates a canvas.
   Open	Opens a pipeline file.
   Save	Saves the pipeline file with its current name.
   Save As	Saves the pipeline plugin flowchart file in a custom path. Click Save to display the Save dialog box. The parameters are described as follows. You can define the parameters as required. Pipeline Filename: name of the pipeline file. Output Path: path for storing the pipeline file.

2. Select plugins in the plugin library, drag the plugins to the editing area, and connect them according to the service process. If you drag an unwanted plugin or make an unwanted connection, select the plugin or connection and press Del to delete it. After the pipeline is drawn, select and right-click all plugins in the pipeline and then choose Set Stream Name from the shortcut menu to set the stream name. If there is more than one pipeline, set the stream name for each pipeline.
   

   By default, Stream name is stream0 and Device Id is 0. Change the values as required. If you specify a value for Device Id, select all plugins that belong to the same stream and set them to the same device.

3. Click Save to save the settings.