Hiva::CreateNnEngine

Description

Creates an NN engine, loads a specified model file, performs inference on the data published by inTopicInfo, and publishes the inference result in outTopicInfo format.

Prototype

uint32_t Hiva::CreateNnEngine(const std::string& engineName, const EngineAttr& engineAttr, const NnTopicInfo& inTopicInfo, const NnTopicInfo& outTopicInfo)

Parameters

Parameter

Input/Output

Description

engineName

Input

Name of the inference engine, which must be unique and cannot be empty. The value cannot exceed 128 bytes and cannot contain backslashes (\) or asterisks (*).

engineAttr

Input

Attributes of the inference engine, including the path, group, and priority of the inference model.

typedef struct EngineAttr_ {           // Define the attributes of the current inference model.
    std::string modelPath;           // Path of the model in the file system
    uint32_t groupId;       // Group to which the model belongs. For details, see . The default value range is 0–1. The value range can be changed, and the maximum value range is 0–3.
    uint32_t  pri;          // Model priority. A smaller value indicates a higher priority. The value ranges from 0 to 7.
};

inTopicInfo

Input

Topic name (vector structure) and queue size of the model input data. All input data has the same queue length. Queue flow control and write overwriting are enabled. The first character of topic name in topicVec must be a slash (/).

outTopicInfo

Output

Topic name (vector structure) and queue size of the model output data. All output data has the same queue length. Queue flow control and write overwriting are disabled. The first character of topic name in topicVec must be a slash (/).

Returns

The return value is of the uint32_t type. If the operation is successful, HIVA_SUCCESS (value being 0) is returned. If other values are returned, the operation fails.

Usage

By invoking this interface, the application can transfer the data published by inTopicInfo to the NN for asynchronous inference and publish the inference result in outTopicInfo format.

Before invoking this interface, the application needs to register inTopicInfo to avoid performing the Publish operation of inTopicInfo before invoking this interface.
After invoking this interface, the application needs to perform the Subscribe operation on outTopicInfo.

typedef struct NnTopicInfo_{
    const std::vector<std::string> topicVec;    
    const uint32_t queueSize;                    // queueSize ranges from 1 to 127.
}NnTopicInfo;

Precautions

Non-reentrant, synchronous interface.
For the data of the same inTopicInfo, you can use this interface to create multiple NNEngines for inference. Ensure that different models and outTopicInfo are used.
The data published by inTopicInfo needs to use the buffer management interface to apply for the shared memory and use the Publish interface to publish the data.
The Subscriber interface needs to be used to subscribe to outTopicInfo.
After NN inference is complete, the data published by inTopicInfo is released. If there are multiple inferences for the same data, the data is released after the last inference is complete.
After NN inference is complete, the callback with outTopicInfo subscribed is invoked. The shared memory pointer of the tensor type is passed to the callback. The shared memory is released by the framework after the callback is complete.
This interface is not a member function of NodeHandle.

Model Groups

Model grouping uses a configuration file (YAML file) to set the attributes of each group. The following is an example of the configuration file. To modify the configuration file, use the CfgMgr interface. For details, see Configuration Management.

AIGroups: 
    group0: 
        aicore_number: 8 
        aivector_number: 4 
        sdma_number: 255     # Default value used
        aicpu_number: 255    # Default value used
        active_sq_number: 1  
    group1: 
        aicore_number: 2 
        aivector_number: 4 
        sdma_number: 255     # Default value used
        aicpu_number: 255    # Default value used
        active_sq_number: 1 		 
    group2: 
        aicore_number: 0     # The value is 0 and other values are invalid.
        aivector_number: 0 
        sdma_number: 255     # Default value used
        aicpu_number: 255    # Default value used
        active_sq_number: 1  
    group3: 
        aicore_number: 0     # The value is 0 and other values are invalid.
        aivector_number: 0 
        sdma_number: 255     # Default value used
        aicpu_number: 255    # Default value used
        active_sq_number: 1 	 

Parent topic: NnEngine