set_tensor

Function

Sets a tensor for a request.

C++ Prototype

void SetTensor(const std::string& tensorName, TensorPtr &tensor)

Python Prototype

set_tensor(tensor_name, tensor)

Parameters

Parameter	Mandatory/Optional	Description	Value
tensor_name	Mandatory	Tensor name.	Valid string type
tensor	Mandatory	Tensor data.	Valid InferTensor class. For details, see InferTensor.

Usage Example

Initialize and construct an inference request ID and inference request based on the string ID. Also, assign a tensor to the request by name.

id = '123'
request_id = llm_manager_python.InferRequestId(id)
infer_request = llm_manager_python.InferRequest(request_id)
name = "INPUT_IDS"
shape = np.array([1,1], dtype=np.int64)
runtime_tensor = llm_manager_python.InferTensor(name, shape, llm_manager_python.InferDataType.TYPE_INT64)
infer_request.set_tensor(name, runtime_tensor)

Parent topic: InferRequest