mxpi_tensorinfer

Function	Infers the input tensor.
Synchronous/Asynchronous (Status)	Synchronous
Constraints	None
Plugin Base Class (Factory)	mxpi_tensorinfer
Input/Output	Input: data type MxpiTensorPackageList (If MxpiVisionList is compatible, MxpiTensorPackageList is automatically converted with three channels.) Output: data type MxpiTensorPackageList
Port Format (Caps)	Static input: {"metadata/tensor"}; dynamic input: {"image/yuv"}. At least one port is required. Multiple ports are allowed. Static output: {"metadata/tensor"}.
Property	For details, see Table 1.

**Table 1** mxpi_tensorinfer plugin properties
Property Name	Description	Mandatory or Not	Modifiable or Not
modelPath	OM file path of the inference model The maximum size of a model is 4 GB. The owner of the model must be the current user, and the permission cannot be higher than 640.	Yes	Yes
dataSource	Index of the input data. Multiple indexes can be configured, but the number of indexes must be the same as that of input ports. The default value is the key value of the output port of the upstream plugin.	No	Yes
outputDeviceId	If the postprocessing .so file is not used, the data is copied from the memory to the position specified by outputDeviceId. To copy the data to the host, set this parameter to -1. To copy the data to the device, set this parameter to the value of deviceId in the stream_config field.	No	Yes
waitingTime	Maximum waiting time that the multi-batch model allows to build a batch. If the actual waiting time exceeds the value of this parameter, the system stops waiting and automatically performs inference. The default value is 5000 ms.	No	Yes
dynamicStrategy	Policy used to select a proper batch size in dynamic batch inference. The default value is Nearest. Nearest: Use the batch size that is closest to the absolute value of the difference between the number of cached images. If the absolute values are the same, use the larger one. Upper: Use the minimum batch size that is greater than or equal to the number of cached images. Lower: Use the maximum batch size that is less than or equal to the number of cached images.	No	Yes
singleBatchInfer	Single-batch inference enablement. The value is of the Boolean type. 0: Selects single-batch or multi-batch inference based on the first dimension of the model. The default value is 0. 1: Selects only single-batch inference regardless of whether the first dimension of the model is 1.	No	Yes
outputHasBatchDim	Whether the model output dimension has the batch dimension. If no, the inference plugin automatically adds the batch dimension to the output tensor. The value is of the Boolean type. 0: No 1: Yes (The default value is 1.)	No	Yes
skipModelCheck	It is under internal debugging. Do not use it.	No	No

Example

After all the data sent by the previous plugins arrives, the plugin enters the Process API (sync mode) and assembles MxpiTensorPackageList (or MxpiVisionList). If the assembled tensor is the same as the model input tensor, inference is enabled and the inference result is output to the output port.

Pipeline sample:

"mxpi_tensorinfer0": {
    "props": {
        "dataSource": "appsrc0,appsrc1,appsrc2",
        "modelPath": "../models/bert/bert.om"
     },
    "factory": "mxpi_tensorinfer",
    "next": "mxpi_classpostprocessor0"
},

Parent topic: Inference Plugins