Development Process

Figure 1 shows the process of using the SDK to develop and run an inference service. In-depth development is an advanced feature of the SDK. You can use it as required.

1. Determine the service process.

   Modularize the service process based on functions such as object detection, image classification, and property recognition. As shown in Figure 1, the service process is divided into image obtaining, image decoding, image resizing, object detection, image cropping, image resizing, image classification, serialization, and result sending.

2. Search for proper plugins.

   Match service functions based on the function description and specifications of existing SDK plugins. For details about the SDK plugins, see Existing Plugin Introduction.

   When the plugins provided by the SDK cannot meet function requirements, you can develop custom plugins by referring to Plugin Development Description, which describes an advanced feature of the SDK.

3. Prepare an inference model.

   Network models (such as MindSpore, Caffe, and TensorFlow) need to be converted using the ATC tool before performing inference on the Ascend AI Processor. For details, see Model Conversion. During model conversion, operator scheduling tuning, weight data rearrangement, and memory usage tuning can be completed. After the model conversion, post-processing is required. In addition, the SDK provides the post-processing dynamic libraries (specified by configuration parameters and loaded and executed by the inference plugin) for common models, such as ResNet-50 and YOLOv3. For details about the post-processing lists of all supported models, see Existing Supported Models.

   When model post-processing cannot meet the requirements, you can develop post-processing dynamic libraries for other models by referring to Model Post-processing (Old Framework), which also describe an advanced feature of the SDK.

4. Orchestrate the process.

   Create a stream configuration file, connect the service stream management module, function plugin, function element, plugin cache, and plugin metadata (see Process Orchestration), and streamline a data flow from input to output of the service (see Data Flow).

5. Integrate the service.

   Call the service stream management API MxStreamManager to initialize the stream, load the service stream configuration file (*.pipeline), obtain the output data from the specified stream based on StreamName in the stream configuration file (see Service Stream Data Interconnection APIs), and destroy the stream.

6. Perform remaining steps.

   Set the environment variable LD_LIBRARY_PATH before compilation (for details, see CMakeLists.txt in the C++ directory in the RUN package). Build the log configuration file logging.conf based on the debugging requirements. Set the operating environment variables MX_SDK_HOME, GST_PLUGIN_SCANNER, and GST_PLUGIN_PATH. Package the plugin dynamic libraries, model files (.om files and the corresponding postprocessing libraries) and stream configuration files, and start the service. For details, see Sample Overview.