Metadata

Metadata is the structured data generated by plugins, such as classification information and object information. It is attached to the plugin buffer for transmission.

Metadata is defined by Protobuf corresponding to APIs in MxpiDataType.pb.h, MxpiDumpData.pb.h, and MxpiOSDType.pb.h files, which are automatically generated by Protobuf and contain user-defined classes in Protobuf. (Do not use them. For details about the proto files, see Metadata Proto File.) The defined data structure is as follows.

Table 1 Data structure of MxpiDataType metadata

Name

Description

Member

MxpiFrame

Stores the description information and data of video/image frames.
  • MxpiFrameInfo frameInfo;
  • MxpiVisionList visionList;

MxpiFrameInfo

Stores the description information about video/image frames, such as the channel ID and frame sequence.
  • uint32 channelId; //Channel ID
  • uint32 frameId; //Frame sequence
  • bool isEos; //Whether the frame is the last frame

MxpiVisionList

Video/Image data list
  • repeated MxpiVision visionVec;

MxpiVision

Video/Image data, including the video/image memory and description information
  • repeated MxpiMetaHeader headerVec;
  • MxpiVisionInfo visionInfo;
  • MxpiVisionData visionData;

MxpiMetaHeader

Information header, which is used to indicate the dependency data of the current data. dataSource is used to find the dependency data list, and memberId is used to find the ID of the dependency data in the list.
  • string parentName; //Data source
  • int32 memberId; //Data ID index
  • string dataSource; //Data source, which is the same as parentName and is recommended.

MxpiMemoryType

Data type enumeration. The data types include the host memory, device memory, and DVPP memory.
  • MXPI_MEMORY_HOST; //Host memory
  • MXPI_MEMORY_DEVICE; //Device memory
  • MXPI_MEMORY_DVPP; // DVPP memory
  • MXPI_MEMORY_HOST_MALLOC; //Use malloc to allocate memory.
  • MXPI_MEMORY_HOST_NEW; //Use new to allocate memory.

MxpiDataType

Data type, including the 8-bit unsigned integer and 32-bit floating point number.
  • MXPI_DATA_TYPE_FLOAT32;
  • MXPI_DATA_TYPE_UINT8;

MxpiVisionInfo

Video/Image data description, such as the image format, width, and height.
  • uint32 format; //Video/Image format
  • uint32 width; //Width
  • uint32 height; //Height
  • uint32 widthAligned; //Width after alignment
  • uint32 heightAligned; //Height after alignment
  • uint32 resizeType; //Image resizing mode
  • float keepAspectRatioScaling; //Resizing ratio
  • repeated MxpiVisionPreProcess preprocessInfo; //Image preprocessing information

MxpiVisionPreProcess

Image preprocessing information, such as the image width and height and coordinates for cropping and pasting an image.
  • uint32 widthSrc; //Image width information
  • uint32 heightSrc; //Image height information
  • uint32 cropLeft; //Left coordinate for image cropping
  • uint32 cropRight; //Right coordinate for image cropping
  • uint32 cropTop; //Top coordinate for image cropping
  • uint32 cropBottom; //Bottom coordinate for image cropping

  • uint32 pasteLeft; //Left coordinate for image pasting
  • uint32 pasteRight; //Right coordinate for image pasting
  • uint32 pasteTop; //Top coordinate for image pasting
  • uint32 pasteBottom; //Bottom coordinate for image pasting

  • uint32 interpolation; //Specified resizing operator
  • string elementName; //Plugin name

MxpiVisionData

Video/Image data content. dataPtr and freeFunc are the uint64 values of the pointer. The pointer must be forcibly converted to this type.
  • uint64 dataPtr //Memory pointer value
  • int32 dataSize; //Memory size, which must be the same as the actual memory size. Otherwise, a core dump may occur.
  • uint32 deviceId; //Device ID
  • MxpiMemoryType memType; //Memory type
  • uint64 freeFunc; //Pointer to the memory destruction function
  • bytes dataStr //Base64 encoding is automatically performed when the bytes data type is serialized into JSON.
  • MxpiDataType dataType; //Data type
  • uint64 matPtr; //Data pointer

MxpiObjectList

Detection object list.
  • repeated MxpiObject objectVec;

MxpiObject

Data structure of the detected object.
  • repeated MxpiMetaHeader headerVec;
  • float x0 //X coordinate of the upper left corner of the object
  • float y0 //Y coordinate of the upper left corner of the object
  • float x1 //X coordinate of the lower right corner of the object
  • float y1 //Y coordinate of the lower right corner of the object
  • repeated MxpiClass classVec //Class information data structure. The internal MxpiMetaHeader is invalid.
  • MxpiImageMask imageMask; //Image semantic segmentation data information

MxpiImageMaskList

Image semantic segmentation data list.
  • repeated MxpiImageMask imageMaskVec;

MxpiImageMask

Image semantic segmentation data information, such as the data class, shape, and type.
  • repeated MxpiMetaHeader headerVec; //Information header of semantic segmentation data
  • repeated string className; //Class information of semantic segmentation data
  • repeated int32 shape; //Shape information of semantic segmentation data
  • int32 dataType; //Semantic segmentation data type
  • bytes dataStr; //Actual semantic segmentation data memory

MxpiClass

Image class data structure.
  • repeated MxpiMetaHeader headerVec;
  • int32 classId; //Class ID
  • string className //Class name
  • float confidence //Class confidence

MxpiClassList

Image class list.
  • repeated MxpiClass classVec;

MxpiAttributeList

Object attribute list.
  • repeated MxpiAttribute attributeVec;

MxpiAttribute

Object attribute.
  • repeated MxpiMetaHeader headerVec;
  • int32 attrId //Attribute ID
  • string attrName; //Attribute class name
  • string attrValue; //Attribute result
  • float confidence; //Attribute confidence

MxpiTrackLetList

List of tracklets.
  • repeated MxpiTrackLet trackLetVec;

MxpiTrackLet

Tracklet.
  • repeated MxpiMetaHeader headerVec;
  • uint32 trackId;
  • uint32 age //Number of alive object frames
  • uint32 hits //Number of object frames that are successfully recorded
  • int32 trackFlag; //Path record status

MxpiTensorPackageList

List of model tensor packages.
  • repeated MxpiTensorPackage tensorPackageVec;

MxpiTensorPackage

Model tensor package.
  • repeated MxpiMetaHeader headerVec;
  • repeated MxpiTensor tensorVec;

MxpiTensor

Data structure of the model tensor
  • uint64 tensorDataPtr; //Memory pointer value
  • int32 tensorDataSize; //Memory size, which must be the same as the actual memory size. Otherwise, a core dump may occur.
  • uint32 deviceId; //Device ID
  • MxpiMemoryType memType; //Memory type
  • uint64 freeFunc; //Pointer to the memory destruction function
  • repeated int32 tensorShape; //Tensor shape
  • bytes dataStr; //Data in the memory
  • int32 tensorDataType; //Data type of the tensor in the memory

MxpiFeatureVectorList

Feature data list.
  • repeated MxpiFeatureVector featureVec;

MxpiFeatureVector

Feature data structure.
  • repeated MxpiMetaHeader headerVec;
  • repeated float featureValues //Feature data

MxpiPoseList

Data list for human pose estimation.
  • repeated MxpiPose poseVec;

MxpiPose

Data information for human pose estimation.
  • repeated MxpiMetaHeader headerVec; //Data header for pose estimation
  • repeated MxpiKeyPoint keyPointVec;

    //Dataset of all key feature points of the human body

  • float score; //Pose estimation confidence

MxpiKeyPoint

Data information of key human feature points.
  • float x; //X coordinate of the key point
  • float y; //Y coordinate of the key point
  • int32 name; //Key point name
  • float score; //Confidence of the key point

MxpiKeyPointAndAngleList

List of key object points and angles.
  • repeated MxpiKeyPointAndAngle keyPointAndAngleVec;

MxpiKeyPointAndAngle

Key object points and angles.
  • repeated MxpiMetaHeader headerVec;
  • repeated float keyPointsVec; //Five key points of the object
  • float angleYaw; //Yaw angle
  • float anglePitch; //Pitch angle
  • float angleRoll; //Roll angle

MxpiTextObjectList

Text object data list.
  • repeated MxpiTextObject objectVec;

MxpiTextObject

Text object data information, such as coordinates, confidence, and text to be detected of the text bounding box
  • repeated MxpiMetaHeader headerVec; //Text object data header
  • float x0; //X coordinate of the lower left corner of the text bounding box
  • float y0; //Y coordinate of the lower left corner of the text bounding box
  • float x1; //X coordinate of the lower right corner of the text bounding box
  • float y1; //Y coordinate of the lower right corner of the text bounding box
  • float x2; //X coordinate of the upper right corner of the text bounding box
  • float y2; //Y coordinate of the upper right corner of the text bounding box
  • float x3; //X coordinate of the upper left corner of the text bounding box
  • float y3; //Y coordinate of the upper left corner of the text bounding box
  • float confidence; //Confidence of the text bounding box
  • string text; //Information about the text to be detected in the bounding box

MxpiTextsInfoList

Text generation data list.
  • repeated MxpiTextsInfo textsInfoVec;

MxpiTextsInfo

Text generation data information.
  • repeated MxpiMetaHeader headerVec; //Text generation data header
  • repeated string text; //Text generation data string

MxpiCustomDataList

User-defined data list.
  • repeated MxpiCustomData dataVec;

MxpiCustomData

User-defined data.
  • repeated MxpiMetaHeader headerVec;
  • map<string, string> map;
Table 2 Data structure of MxpiDumpData metadata

Name

Description

Member

MxpiDumpData

Saves the MxpiBuffer content.
  • Buffer buffer;
  • repeated MetaData metaData;

Buffer

Buffer data in MxpiBuffer.
  • bytes bufferData

MetaData

Metadata in MxpiBuffer
  • string key; //Key value of metadata
  • string content; //Protobuf data after JSON serialization
  • string protoDataType; //Protobuf data type
Table 3 Data structure of MxpiOSDType metadata

Name

Description

Member

MxpiOsdInstancesList

Stores the object or class OSD list.
  • repeated MxpiOsdInstances osdInstancesVec;

MxpiOsdInstances

Stores the OSD attribute descriptions.
  • repeated MxpiMetaHeader headerVec;
  • repeated MxpiOsdText osdTextVec; //Text description
  • repeated MxpiOsdLine osdLineVec; //Line description
  • repeated MxpiOsdRect osdRectVec; //Rectangle description
  • repeated MxpiOsdCircle osdCircleVec; //Circle description

MxpiOsdText

Stores the text attribute descriptions.
  • repeated MxpiMetaHeader headerVec;
  • string text; //Text to be added
  • int32 x0; //Add the X coordinate of the text origin.
  • int32 y0; //Add the Y coordinate of the text origin.
  • int32 fontFace; //Font type of the text
  • double fontScale; //Font size of the text
  • bool bottomLeftOrigin; //The value true indicates that the coordinate origin is the coordinate at the upper left corner of the text. The value false indicates that the coordinate origin is the coordinate at the lower left corner.
  • MxpiOsdParams osdParams; // Instance objects of the OSD public properties
  • bool fixedArea; // Whether the OSD attribute descriptions are scaled based on the original image proportion.

MxpiOsdLine

Stores the line attribute descriptions.
  • repeated MxpiMetaHeader headerVec;
  • int32 x0; //X coordinate of the upper left corner of the line
  • int32 y0; //Y coordinate of the upper left corner of the line
  • int32 x1; //X coordinate of the lower right corner of the line
  • int32 y1; //Y coordinate of the lower right corner of the line
  • MxpiOsdParams osdParams; // Instance objects of the OSD public properties

MxpiOsdRect

Stores the rectangle attribute descriptions.
  • repeated MxpiMetaHeader headerVec;
  • int32 x0; //X coordinate of the upper left corner of the rectangle
  • int32 y0; //Y coordinate of the upper left corner of the rectangle
  • int32 x1; //X coordinate of the lower right corner of the rectangle
  • int32 y1; //Y coordinate of the lower right corner of the rectangle
  • MxpiOsdParams osdParams; // Instance objects of the OSD public properties
  • bool fixedArea; // Whether the OSD attribute descriptions are scaled based on the original image proportion.

MxpiOsdCircle

Stores the circle attribute descriptions.
  • repeated MxpiMetaHeader headerVec;
  • int32 x0; //X coordinate of the upper left corner of the circle
  • int32 y0; //Y coordinate of the upper left corner of the circle
  • int32 radius; //Radius of the circle
  • MxpiOsdParams osdParams; // Whether the OSD attribute descriptions are scaled based on the original image proportion.

MxpiOsdParams

Stores the descriptions about OSD public properties.
  • uint32 scalorB; //Value of color channel B. The value range is [0, 255].
  • uint32 scalorG; //Value of color channel G. The value range is [0, 255].
  • uint32 scalorR; //Value of color channel R. The value range is [0, 255].
  • int32 thickness; //Thickness
  • int32 lineType; //Line type
  • int32 shift; //Scaling-down parameter

For details about the line types and font types in Table 3, see Table 1.

Parent topic: Process Orchestration Data Structures