Sample Code for Executing a Dynamic-Shape Operator

This section describes the key APIs and sample code for calling dynamic-shape operators in single-operator model execution mode.

Principles

For operators that support dynamic shape:

If the output shape of an operator is known, the basic execution process of this operator is similar to that of a static-shape operator. For details about the API call sequence, see Overview. For details about the sample code for executing a static-shape operator, see Sample Code for Executing a Static-Shape Operator.
If the output shape of an operator is unknown, you need to call aclopInferShape, aclGetTensorDescNumDims, aclGetTensorDescDimV2, and aclGetTensorDescDimRange to infer or estimate the output shape of the operator as the input of the operator execution API aclopExecuteV2.
When calling aclopInferShape, determine the allocation of memory for storing the input tensor data of the operator based on the application run mode. If the application runs on the host, allocate host memory. If the application runs on the device, allocate device memory.

Sample Code

The following is a code snippet of key steps only, which is not ready to be built or run. After APIs are called, you need to add exception handling branches and record error logs and info logs.

// ...

const char *opType;
int numInputs;
aclTensorDesc *inputDesc[2];
aclDataBuffer *inputs[2];
int numOutputs;
aclTensorDesc *outputDesc[1];
aclopAttr *attr;

aclError ret = aclopInferShape(opType, numInputs, inputDesc, inputs, numOutputs, outputDesc, attr);

std::vector<std::vector<int64_t>> tensorDims; //Tensor shape output by InferShape
// Perform a for loop for each output to infer or estimate the shape value.
for (int index = 0; index < numOutputs; ++index) {
	std::vector<int64_t> dimSize; //Output shape
	size_t dimNums = aclGetTensorDescNumDims(outputDesc[index]);
        // Reserved. The dimension count in the dynamic-shape scenario is unknown.
	if (dimNums == ACL_UNKNOWN_RANK)  {
		// The user estimates the maximum shape value (maxshape).
		dimSize.push_back(max_shape);
	} else {
		for (size_t i = 0; i < dimNums; ++i) {
		    int64_t dim;
			ret = aclGetTensorDescDimV2(outputDesc[index], i, &dim);

                        // The dimensions are dynamic in the dynamic-shape scenario.
			if(dim == -1) {
			    int64_t dimRange[2];
                                // Obtain the shape range and use the maximum shape value to construct the output tensor description, as an argument of the aclopExecuteV2 call.
				ret = aclGetTensorDescDimRange(outputDesc[index], i, 2, dimRange);
				dim = dimRange[1];
			}
			dimSize.push_back(dim);
		}
	}
	tensorDims.push_back(dimSize);
}

// Construct the operator input tensor description and input tensors, as the input parameters of the aclopExecuteV2 call.
aclTensorDesc *inputDescNew[2];
aclDataBuffer *inputsNew[2];
aclDataBuffer *outputsNew[1];
// The preceding provides the operator output shape and the output tensor description (that is, the value of outputDescNew) is constructed based on dims in tensorDims, as the input parameters of the aclopExecuteV2 call.
ret = aclopExecuteV2(opType, numInputs, inputDescNew, inputsNew, numOutputs, outputDescNew, outputsNew, attr, stream);

// Add the following code lines to obtain the accurate shape after operator execution.
// Perform a for loop for the tensorDesc of each output.
std::vector<std::vector<int64_t>> outTensorDims; // Accurate output tensor shape
for (int index = 0; index < numOutputs; ++index) {
    std::vector<int64_t> dimSize;
	int dimNums = aclGetTensorDescNumDims(outputDescNew[index]);
	for (int i = 0; i < dimNums; i++){
		int64_t dim;
		ret = aclGetTensorDescDimV2(outputDescNew[index], i, &dim);
		dimSize.push_back(dim);
	}
	outTensorDims.push_back(dimSize);
}
// ...

Parent topic: Single-Operator Model Execution