Documentation

Metadata

Metadata is the structured data generated by plugins, such as classification information and object information. Metadata is attached to the plugin buffer for transmission.

Metadata is defined by protobuf (corresponding to the API in the MxpiDataType.pb.h, MxpiDumpData.pb.h, and MxpiOSDType.pb.h files, which are automatically generated by protobuf). For details about the protobuf file, see Metadata Proto File. The defined data structure is as follows:

Table 1 Data structure of MxpiDataType Metadata

Name

Function

Members

MxpiFrame

Stores the description and data of video or image frames.
  • MxpiFrameInfo frameInfo
  • MxpiVisionList visionList

MxpiFrameInfo

Stores the description information about video or image frames, such as the channel ID and frame sequence.
  • uint32 channelId; // Channel ID
  • uint32 frameId; // Frame sequence
  • bool isEos; // Whether the frame is the last frame

MxpiVisionList

Video/Image data list
  • repeated MxpiVision visionVec

MxpiVision

Video/Image data, including the video/image memory and description information
  • repeated MxpiMetaHeader headerVec
  • MxpiVisionInfo visionInfo
  • MxpiVisionData visionData

MxpiMetaHeader

Information header, which is used to indicate the dependency data of the current data. dataSource is used to find the dependency data list, and memberId is used to find the ID of the dependency data in the list.
  • string dataSource; // Index name of the dependency data list
  • int32 memberId; // Number in the dependency data list

MxpiMemoryType

Data type enumeration. The data types include the host memory, device memory, and DVPP memory.
  • MXPI_MEMORY_HOST; //Host memory
  • MXPI_MEMORY_DEVICE; //Device memory
  • MXPI_MEMORY_DVPP; // DVPP memory
  • MXPI_MEMORY_HOST_MALLOC; // Use mallloc to allocate memory.
  • MXPI_MEMORY_HOST_NEW; // Use new to allocate memory.

MxpiDataType

Data type, including the 8-bit unsigned integer and 32-bit floating point number.
  • MXPI_DATA_TYPE_FLOAT32;
  • MXPI_DATA_TYPE_UINT8;

MxpiVisionInfo

Video/Image data description, such as the image format, width, and height.
  • uint32 format; //Video/Image format
  • uint32 width; //Width
  • uint32 height; //Height
  • uint32 widthAligned; // Width after alignment
  • uint32 heightAligned; // Height after alignment
  • uint32 resizeType; //Image resizing mode
  • float keepAspectRatioScaling; // Resizing ratio
  • repeated MxpiVisionPreProcess preprocessInfo; // Image pre-processing information

MxpiVisionPreProcess

Image pre-processing information, such as the image width and height and coordinates for cropping and pasting an image.
  • uint32 widthSrc; // Image width information
  • uint32 heightSrc; // Image height information
  • uint32 cropLeft; // Left coordinate for image cropping
  • uint32 cropRight; // Right coordinate for image cropping
  • uint32 cropTop; // Top coordinate for image cropping
  • uint32 cropBottom; // Bottom coordinate for image cropping
  • uint32 pasteLeft; // Left coordinate for image pasting
  • uint32 pasteRight; // Right coordinate for image pasting
  • uint32 pasteTop; // Top coordinate for image pasting
  • uint32 pasteBottom; // Bottom coordinate for image pasting
  • uint32 interpolation; // Specified resizing operator
  • string elementName; // Plugin name

MxpiVisionData

Video/Image data content. dataPtr and freeFunc are the uint64 values of the pointer. The pointer must be forcibly converted to this type.
  • uint64 dataPtr //Memory pointer value
  • int32 dataSize; // Memory size, which must be the same as the actual memory size. Otherwise, a core dump may occur.
  • uint32 deviceId; // Device ID
  • MxpiMemoryType memType; // Memory type
  • uint64 freeFunc; // Pointer to the memory destruction function
  • bytes dataStr // Base64 encoding is automatically performed when the bytes data type is serialized into JSON.
  • MxpiDataType dataType; // Data type
  • uint64 matPtr; // Data pointer

MxpiObjectList

Detection object list
  • repeated MxpiObject objectVec

MxpiObject

Detected object data structure
  • repeated MxpiMetaHeader headerVec
  • float x0 // X coordinate of the upper left corner of the object
  • float y0 // Y coordinate of the upper left corner of the object
  • float x1 // X coordinate of the lower right corner of the object
  • float y1 // Y coordinate of the lower right corner of the object
  • repeated MxpiClass classVec // Class information data structure. The internal MxpiMetaHeader is invalid.
  • MxpiImageMask imageMask; // Image semantic segmentation data information

MxpiImageMaskList

Image semantic segmentation data list.
  • repeated MxpiImageMask imageMaskVec;

MxpiImageMask

Image semantic segmentation data information, such as data type, shape, and class.
  • repeated MxpiMetaHeader headerVec; // Information header of semantic segmentation data
  • repeated string className; // Class information of semantic segmentation data
  • repeated int32 shape; // Shape information of semantic segmentation data
  • int32 dataType; // Semantic segmentation data type
  • bytes dataStr; // Actual semantic segmentation data memory

MxpiClass

Image class data structure
  • repeated MxpiMetaHeader headerVec
  • int32 classId; // Class ID
  • string className // Class name
  • float confidence // Class confidence level

MxpiClassList

Image class list
  • repeated MxpiClass classVec

MxpiAttributeList

Object attribute list
  • repeated MxpiAttribute attributeVec

MxpiAttribute

Object attribute
  • repeated MxpiMetaHeader headerVec
  • int32 attrId // Attribute ID
  • string attrName; // Attribute class name
  • string attrValue; // Attribute result
  • float confidence; // Attribute confidence level

MxpiTrackLetList

List of tracklets
  • repeated MxpiTrackLet trackLetVec

MxpiTrackLet

Tracklet
  • repeated MxpiMetaHeader headerVec
  • uint32 trackId
  • uint32 age // Number of alive object frames
  • uint32 hits // Number of object frames that are successfully recorded
  • int32 trackFlag; // Path record status

MxpiTensorPackageList

List of model tensor packages
  • repeated MxpiTensorPackage tensorPackageVec

MxpiTensorPackage

Model tensor package
  • repeated MxpiMetaHeader headerVec
  • repeated MxpiTensor tensorVec

MxpiTensor

Data structure of the model tensor
  • uint64 tensorDataPtr; // Memory pointer value
  • int32 tensorDataSize; // Memory size, which must be the same as the actual memory size. Otherwise, a core dump may occur.
  • uint32 deviceId; // Device ID
  • MxpiMemoryType memType; // Memory type
  • uint64 freeFunc; // Pointer to the memory destruction function
  • repeated int32 tensorShape; // Tensor shape
  • bytes dataStr; // Data in the memory
  • int32 tensorDataType; // Data type of the tensor in the memory

MxpiFeatureVectorList

Feature data list
  • repeated MxpiFeatureVector featureVec

MxpiFeatureVector

Feature data structure
  • repeated MxpiMetaHeader headerVec
  • repeated float featureValues // Feature data

MxpiPoseList

Data list for human pose estimation
  • repeated MxpiPose poseVec;

MxpiPose

Data information for human pose estimation
  • repeated MxpiMetaHeader headerVec; // Data header for pose estimation
  • repeated MxpiKeyPoint keyPointVec;

    // Dataset of all key feature points of the human body

  • float score; // Pose estimation confidence

MxpiKeyPoint

Data information of key human feature points
  • float x; // X coordinate of the key point
  • float y; // Y coordinate of the key point
  • int32 name; // Key point name
  • float score; // Confidence of the key point

MxpiKeyPointAndAngleList

List of key object points and angles
  • repeated MxpiKeyPointAndAngle keyPointAndAngleVec

MxpiKeyPointAndAngle

Key object points and angles
  • repeated MxpiMetaHeader headerVec;
  • repeated float keyPointsVec; // Five key points of the object
  • float angleYaw; // Yaw angle
  • float anglePitch; // Pitch angle
  • float angleRoll; // Roll angle

MxpiTextObjectList

Text object data list
  • repeated MxpiTextObject objectVec;

MxpiTextObject

Text object data information, such as coordinates of the text bounding box, confidence, and text to be detected
  • repeated MxpiMetaHeader headerVec; // Text object data header
  • float x0; // X coordinate of the lower left corner of the text bounding box
  • float y0; // Y coordinate of the lower left corner of the text bounding box
  • float x1; // X coordinate of the lower right corner of the text bounding box
  • float y1; // Y coordinate of the lower right corner of the text bounding box
  • float x2; // X coordinate of the upper right corner of the text bounding box
  • float y2; // Y coordinate of the upper right corner of the text bounding box
  • float x3; // X coordinate of the upper left corner of the text bounding box
  • float y3; // Y coordinate of the upper left corner of the text bounding box
  • float confidence; // Confidence of the text bounding box
  • string text; // Information about the text to be detected in the bounding box

MxpiTextsInfoList

Text generation data list
  • repeated MxpiTextsInfo textsInfoVec;

MxpiTextsInfo

Text generation data information
  • repeated MxpiMetaHeader headerVec; // Text generation data header
  • repeated string text; // Text generation data string
Table 2 Data structure of MxpiDumpData Metadata

Name

Function

Members

MxpiDumpData

Saves the MxpiBuffer content.
  • Buffer buffer
  • repeated MetaData metaData

Buffer

Buffer data in the MxpiBuffer
  • bytes bufferData

MetaData

metaData in the MxpiBuffer
  • string key; // Key value of metaData
  • string content; // Protobuf data after JSON serialization
  • string protoDataType; // Protobuf data type
Table 3 Data structure of MxpiOSDType Metadata

Name

Function

Members

MxpiOsdInstancesList

Stores the object or class OSD list.
  • repeated MxpiOsdInstances osdInstancesVec;

MxpiOsdInstances

Stores the OSD attribute descriptions.
  • repeated MxpiMetaHeader headerVec
  • repeated MxpiOsdText osdTextVec; // Text description
  • repeated MxpiOsdLine osdLineVec; // Line description
  • repeated MxpiOsdRect osdRectVec; // Rectangle description
  • repeated MxpiOsdCircle osdCircleVec; // Circle description

MxpiOsdText

Stores the text attribute descriptions.
  • repeated MxpiMetaHeader headerVec
  • string text; // Text to be added
  • int32 x0; // Add the X coordinate of the text origin.
  • int32 y0; // Add the Y coordinate of the text origin.
  • int32 fontFace; // Font type of the text
  • double fontScale; // Font size of the text
  • bool bottomLeftOrigin; // The value true indicates that the coordinate origin is the coordinate at the upper left corner of the text. The value false indicates that the coordinate origin is the coordinate at the lower left corner.
  • MxpiOsdParams osdParams; // Instance objects of the OSD public attributes
  • bool fixedArea; // Indicates whether the OSD attribute descriptions are scaled based on the original image proportion.

MxpiOsdLine

Stores the line attribute descriptions.
  • repeated MxpiMetaHeader headerVec
  • int32 x0; // X coordinate of the upper left corner of the line
  • int32 y0; // Y coordinate of the upper left corner of the line
  • int32 x1; // X coordinate of the lower right corner of the line
  • int32 y1; // Y coordinate of the lower right corner of the line
  • MxpiOsdParams osdParams; // Instance objects of the OSD public attributes

MxpiOsdRect

Stores the rectangle attribute descriptions.
  • repeated MxpiMetaHeader headerVec
  • int32 x0; // X coordinate of the upper left corner of the rectangle
  • int32 y0; // Y coordinate of the upper left corner of the rectangle
  • int32 x1; // X coordinate of the lower right corner of the rectangle
  • int32 y1; // Y coordinate of the lower right corner of the rectangle
  • MxpiOsdParams osdParams; // Instance objects of the OSD public attributes
  • bool fixedArea; // Indicates whether the OSD attribute descriptions are scaled based on the original image proportion.

MxpiOsdCircle

Stores the circle attribute descriptions.
  • repeated MxpiMetaHeader headerVec
  • int32 x0; // X coordinate of the upper left corner of the circle
  • int32 y0; // Y coordinate of the upper left corner of the circle
  • int32 radius; // Radius of the circle
  • MxpiOsdParams osdParams; // Indicates whether the OSD attribute descriptions are scaled based on the original image proportion.

MxpiOsdParams

Stores the descriptions about OSD public attributes.
  • uint32 scalerB; // Value of color channel B. The value ranges from 0 to 255.
  • uint32 scalerG; // Value of color channel G. The value ranges from 0 to 255.
  • uint32 scalerR; // Value of color channel R. The value ranges from 0 to 255.
  • int32 thickness; // Thickness
  • int32 lineType; // Line type
  • int32 shift; // Scaling parameter

For details about the line types and font types in Table 3, see Table 2.

Parent topic: Data Structure