mxpi_tensorinfer

Function	Infers the input tensor.
Synchronous/Asynchronous (Status)	Synchronous
Constraints	None
Plugin Base Class (Factory)	mxpi_tensorinfer
Input/Output	Input: data type MxpiTensorPackageList (If MxpiVisionList is compatible, it is automatically converted to MxpiTensorPackageList with three channels.) Output: data type MxpiTensorPackageList
Port Format (Caps)	Static input: {"metadata/tensor"}; dynamic input: {"image/yuv"}. At least one port is required. Multiple ports are allowed. Static output: {"metadata/tensor"}.
Property	For details, see Table 1.

**Table 1** mxpi_tensorinfer plugin properties
Property Name	Description	Mandatory or Not	Modifiable or Not
modelPath	OM file path of the inference model. The maximum size of the model is 4 GB. The owner of the model must be the current user, and the permission cannot be higher than 640.	Yes	Yes
outputDeviceId	If the postprocessing .so file is not used, the data is copied from the memory to the position specified by outputDeviceId. To copy the data to the host, set this parameter to -1. To copy the data to the device, set this parameter to the value of deviceId in the stream_config field.	No	Yes
waitingTime	Maximum waiting time that the multi-batch model allows to build a batch. If the actual waiting time exceeds the value of this parameter, the system stops waiting and automatically performs inference. The default value is 5000ms.	No	Yes
dynamicStrategy	Policy used to select a proper batch size in dynamic batch inference. The default value is Nearest. Nearest: Uses the batch size that is closest to the absolute value of the difference between the number of cached images. If the absolute values are the same, use the larger one. Upper: Uses the minimum batch size that is greater than or equal to the number of cached images. Lower: Uses the maximum batch size that is less than or equal to the number of cached images.	No	Yes
singleBatchInfer	Single-batch inference switch. The value is of the Boolean type. The default value is 0. 0: Selects single-batch or multi-batch inference based on the first dimension of the model. 1: Selects only single-batch inference regardless of whether the first dimension of the model is 1.	No	Yes
outputHasBatchDim	Specifies whether the model output dimensions include the batch dimension. If no, the inference plugin automatically adds the batch dimension to the output tensor. The value is of the Boolean type and the default value is 1. 0: No 1: Yes	No	Yes
skipModelCheck	Skips input verification of model data.	No	No

Example

After all the data sent by the previous plugins arrives, the plugin enters the Process API (sync mode) and assembles MxpiTensorPackageList (or MxpiVisionList). If the assembled tensor is the same as the model input tensor, inference is enabled and the inference result is output to the output port.

Pipeline sample:

      
           "mxpi_tensorinfer0": {
    "props": {
        "dataSource": "appsrc0,appsrc1,appsrc2",
        "modelPath": "../models/bert/bert.om"
     },
    "factory": "mxpi_tensorinfer",
    "next": "mxpi_classpostprocessor0"
},

When the inference result is used for precision test, the preprocessing modes before model inference must be the same as the modes used during model training, including but not limited to the resizing mode, interpolation mode during resizing, image cropping mode, and alignment mode.

Parent topic: Inference Plugins