SetReqType interface

Function

Sets the request type, which can be prefill or decode.

Format

void SetReqType(mindie_llm::InferReqType reqType);

Parameters

Usage Example

After constructing a request object, set the request type for the request object.

1
2
3

mindie_llm::InferRequestId runtimeReqId(0);
std::shared_ptr<mindie_llm::InferRequest> runtimeRequest(runtimeReqId);
runtimeRequest->SetReqType(mindie_llm::InferReqType::REQ_STAND_INFER);

Return Value

N/A