LlmManager

Function

Default constructor

Format

LlmManager(const std::string &llmConfigPath, mindie_llm::GetRequestsCallback getRequest,
        mindie_llm::SendResponsesCallback sendResponse, mindie_llm::ControlSignalCallback controlCallback,
        mindie_llm::LlmManagerStatsCallback statusCallback,
        mindie_llm::SendStatusResponseCallback statusResponseCallback,
std::map<std::string, std::string> ipInfo = std::map<std::string, std::string>())

Parameters

The following table describes the five function pointer types and parameters defined by Callback in Functions.

Parameter	Mandatory/Optional	Description	Value
ipInfo	Mandatory	Passes the parameters related to the node identity information required for model initialization in the PD scenario to the model.	std::map<std::string, std::string>
llmConfigPath	Mandatory	Passes the configuration path required by LlmManager to the constructor of LlmManager.	const std::string

Usage Example

Construct an LlmManager instance.

mindie_llm::GetRequestsCallback getRequestCallback;
mindie_llm::SendResponsesCallback sendResponsesCallback;
mindie_llm::ControlSignalCallback stopSignalCallback;
mindie_llm::LlmManagerStatsCallback statsCallback;
mindie_llm::SendStatusResponseCallback sendStatusCallback;
const std::string llmConfigPath;
std::map<std::string, std::string> ipInfo;
std::shared_ptr<mindie_llm::LlmManager> llmManager = std::make_shared<mindie_llm::LlmManager>(llmConfigPath, getRequestCallback, sendResponsesCallback, stopSignalCallback, statsCallback, sendStatusCallback,ipInfo);

Return Value

LlmManager object

Parent topic: LlmManager