Graph Build to an Offline Model

Overview

You can set a range of dynamic dimension size profiles in ND format at model build time. Applies to the scenario where the dimensions for inference are unfixed.

Procedure

1. In Data operator definition, set the dynamic dimension in the shape to -1.

   1 2 3 4 5

   auto shape_data = vector<int64_t>({ 1,-1,-1 }); TensorDesc desc_data(ge::Shape(shape_data), FORMAT_ND, DT_FLOAT); auto data = op::Data("data"); data.update_input_desc_data(desc_data); data.update_output_desc_out(desc_data);

2. At model build time, set INPUT_SHAPE and INPUT_FORMAT in the options argument passed to the aclgrphBuildModel call and specify the dynamic dimension size profiles by setting DYNAMIC_DIMS.
   

   • INPUT_FORMAT must be consistent with the format of each Data operator, and only ND is supported. Otherwise, model build fails.
   • INPUT_SHAPE must be set.

   1 2 3 4 5 6 7

   void PrepareOptions(std::map<std::string, std::string>& options) { options.insert({ {ge::ir_option::INPUT_FORMAT, "ND"}, {ge::ir_option::INPUT_SHAPE, "data:1,-1,-1"}, {ge::ir_option::DYNAMIC_DIMS, "1,2;3,4;5,6;7,8"} // At model build time, set the shape of Data operator as "1,1,2; 1,3,4; 1,5,6; 1,7,8". }); }

Precautions

• This feature is exclusive with dynamic batch_size, dynamic image size, and AIPP. Only one of them can be enabled at a time.
• The format is "dim1,dim2,dim3;dim4,dim5,dim6;dim7,dim8,dim9". Enclose all profiles in double quotation marks ("") and separate each profile with semicolons (;). The dim values match the –1 dimensions in the INPUT_SHAPE argument, with ordering preserved. For example:

  1 2 3 4 5 6 7

  void PrepareOptions(std::map<std::string, std::string>& options) { options.insert({ {ge::ir_option::INPUT_FORMAT, "ND"}, {ge::ir_option::INPUT_SHAPE, "data:1,1,40,-1;label:1,-1;mask:-1,-1"}, {ge::ir_option::DYNAMIC_DIMS, "20,20,1,1;40,40,2,2;80,60,4,4"} }); }

  The supported input shapes in model building are as follows:

  Profile 0: data(1,1,40,20)+label(1,20)+mask(1,1)

  Profile 1: data(1,1,40,40)+label(1,40)+mask(2,2)

  Profile 2: data(1,1,40,80)+label(1,60)+mask(4,4)

• If this option is used to set the dynamic dimensions during model build, you need to perform the following operations before calling the model execution APIs to run an application project for inference:
  • Use the aclmdlSetInputDynamicDims API to set the real dimensions.
  • If aclmdlSetInputDynamicDims is not called, the maximum value within the dynamic dimension range is assigned by default during model execution.

  For details about the APIs, see "aclmdlSetInputDynamicDims" .