InferTensor

Function

Constructor

C++ Prototype

InferTensor() = default;
InferTensor(std::string name, InferDataType dataType, std::vector<int64_t> dataShape);

Python Prototype

InferTensor()
InferTensor(name, data_type, data_shape)

Parameters

Parameter	Mandatory/Optional	Description	Value
name	Mandatory	Tensor name.	Valid string type
data_type	Mandatory	Tensor data type	Valid data type
data_shape	Mandatory	Tensor data shape	Valid int list that fits within the C++ int64 range.

Usage Example

Construct a runtime tensor.

runtime_tensor1 = llm_manager_python.InferTensor()
name = "INPUT_IDS"
shape = np.array([1,1], dtype=np.int64)
runtime_tensor2 = llm_manager_python.InferTensor(name, llm_manager_python.InferDataType.TYPE_INT64, shape)

Parent topic: InferTensor