Init interface

Function

Initializes the LlmManager.

Format

Status Init(uint32_t modelInstanceId, std::set<size_t> npuDeviceIds)
Status Init(uint32_t modelInstanceId, std::set<size_t> npuDeviceIds, std::map<std::string, std::string> extendInfo)

Parameters

Parameter	Mandatory/Optional	Description	Value
modelInstanceId	Mandatory	Model instance ID.	uint32_t type. The value must be greater than 0 and defaults to 0. The multi-model instance function is supported.
npuDeviceIds	Mandatory	NPU device IDs.	std::set<size_t> .
extendInfo	Optional	Additional configurations provided for initialization.	std::map<std::string, std::string>

Usage Example

Constructs an LlmManager instance and initializes it.

// Define the function in 6.1.1.4.
mindie_llm::GetRequestsCallback getRequestCallback;
mindie_llm::SendResponsesCallback sendResponsesCallback;
mindie_llm::ControlSignalCallback stopSignalCallback;
mindie_llm::LlmManagerStatsCallback statsCallback;
mindie_llm::SendStatusResponseCallback sendStatusCallback;
const std::string llmConfigPath;
std::map<std::string, std::string> ipInfo;
std::shared_ptr<mindie_llm::LlmManager> llmManager = std::make_shared<mindie_llm::LlmManager>(llmConfigPath, getRequestCallback, sendResponsesCallback, stopSignalCallback, statsCallback, sendStatusCallback,ipInfo);
uint32_t modelInstanceId = 0;
std::set<size_t> npuDeviceIds = {0};
llmManager->Init(modelInstanceId,npuDeviceIds); 

Return Value

Status object.

Parent topic: LlmManager