Documentation

Executing a Dynamic-Shape Operator (No Operator Selector Registered)

This section describes the key APIs and sample code for calling dynamic-shape operators in single-operator model execution mode.

Principles

For operators that support dynamic shape:

Sample Code

After APIs are called, you need to add exception handling branches and record error logs and info logs. The following is a code snippet of key steps only, which is not ready to be built or run.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
 
// ......

const char *opType;
int numInputs;
aclTensorDesc *inputDesc[2];
aclDataBuffer *inputs[2];
int numOutputs;
aclTensorDesc *outputDesc[1];
aclopAttr *attr;

//Determine the memory allocation method based on the app run mode. If the app runs on the host, allocate host memory. If the app runs on the device, allocate device memory.
aclError ret = aclopInferShape(opType, numInputs, inputDesc, inputs, numOutputs, outputDesc, attr);

std::vector<std::vector<int64_t>> tensorDims; //Tensor shape output by InferShape
//Perform a for loop for each output to infer or estimate the shape value.
for (int index = 0; index < numOutputs; ++index) {
	std::vector<int64_t> dimSize; //Output shape
	size_t dimNums = aclGetTensorDescNumDims(outputDesc[index]);
        //Reserved. The dimension count in the dynamic-shape scenario is unknown.
	if (dimNums == ACL_UNKNOWN_RANK)  {
		//The user estimates the maximum shape value (maxshape).
		dimSize.push_back(max_shape);
	} else {
		for (size_t i = 0; i < dimNums; ++i) {
		    int64_t dim;
			ret = aclGetTensorDescDimV2(outputDesc[index], i, &dim);

                        //The dimensions are dynamic in the dynamic-shape scenario.
			if(dim == -1) {
			    int64_t dimRange[2];
                                //Obtain the shape range and use the maximum shape value to construct the output tensor description, as an argument of the aclopExecuteV2 call.
				ret = aclGetTensorDescDimRange(outputDesc[index], i, 2, dimRange);
				dim = dimRange[1];
			}
			dimSize.push_back(dim);
		}
	}
	tensorDims.push_back(dimSize);
}

//Construct the operator input tensor description and input tensors, as the input parameters of the aclOpExecuteV2 call.
aclTensorDesc *inputDescNew[2];
aclDataBuffer *inputsNew[2];
aclDataBuffer *outputsNew[1];
//The preceding provides the operator output shape and the output tensor description (that is, the value of outputDescNew) is constructed based on dims in tensorDims, as the input parameters of the aclopExecuteV2 call.
ret = aclopExecuteV2(opType, numInputs, inputDescNew, inputsNew, numOutputs, outputDescNew, outputsNew, attr, stream);

//Add the following code lines to obtain the accurate shape after operator execution.
//Perform a for loop for the tensorDesc of each output.
std::vector<std::vector<int64_t>> outTensorDims; //Accurate output tensor shape
for (int index = 0; index < numOutputs; ++index) {
    std::vector<int64_t> dimSize;
	int dimNums = aclGetTensorDescNumDims(outputDescNew[index]);
	for (int i = 0; i < dimNums; i++){
		int64_t dim;
		ret = aclGetTensorDescDimV2(outputDescNew[index], i, &dim);
		dimSize.push_back(dim);
	}
	outTensorDims.push_back(dimSize);
}
// ......