Adding Dynamic Dimensions Support to an Offline Model

Overview

To support scenarios such as Transformer using dynamic tensor dimensions, the --dynamic_dims option provided by ATC allows you to set dynamic dimension profiles in ND format. ND: any format.

Procedure

This section takes the TensorFlow ResNet50 network model for illustration. Click here to download the model file (*.pb) of the ResNet50 network.

1. Log in to the development environment as the CANN running user and upload the model file to any directory in the development environment, for example, $HOME/module/.
2. Run the following command (the path and file arguments in the command are for reference only):

   atc --model=$HOME/module/resnet50_tensorflow_1.7.pb --framework=3 --output=$HOME/module/out/tf_resnet50 --soc_version=<soc_version>  --input_shape="Placeholder:-1,-1,-1,3" --dynamic_dims="1,224,224;8,448,448" --input_format=ND

   The key options are described as follows:

   • --dynamic_dims: sets dynamic dimension profiles in ND format.
   • --input_shape: specifies the shape of the model input data. This option is used together with --dynamic_dims.
   • --input_format: specifies the input format. The value is ND here.
   • --model: specifies the path of the ResNet50 model file.
   • --framework: specifies the original framework type. The value 3 indicates TensorFlow.
   For details about the options, see Command Line Options. If the following information is displayed, the model conversion is successful. If the model conversion fails, locate the fault by referring to "Error Codes".

   1	ATC run success, welcome to the next use.

   Find the generated offline model in the path specified by the --output argument.

   In your OM offline model, you will find a newly added input (for example, Data in the red outline in Figure 1), which provides specific dimensions for model inference. Assume that the dimensions of input a are dynamic. In the generated OM offline model, input b is added to describe the dimensions of input a.

   Figure 1 Offline model with dynamic dimensions