add_tensor

Function

Adds a tensor.

C++ Prototype

Status AddTensor(const std::string& tensorName, TensorPtr &tensor)

Python Prototype

add_tensor(tensor_name, tensor)

Parameters

Usage Example

Initialize and construct an inference request ID and inference request based on the string ID. Also, allocate a tensor to the request based on the name and shape, and add the tensor to the corresponding request.

1
2
3
4
5
6
7

id = '123'
request_id = llm_manager_python.InferRequestId(id)
infer_request = llm_manager_python.InferRequest(request_id)
name = "INPUT_IDS"
shape = np.array([1,1], dtype=np.int64)
runtime_tensor = llm_manager_python.InferTensor(name, shape, llm_manager_python.InferDataType.TYPE_INT64)
infer_request.add_tensor(name, runtime_tensor)