Documentation

Graph Build to an Offline Model

You can specify a range of dynamic dimension size profiles in APIs to support dynamic-shape inputs. This section describes the details.

Overview

You can set a range of dynamic dimension size profiles in ND format at model build time. Applies to the scenario where the dimensions for inference are unfixed.

Applicability

Atlas 200/300/500 Inference Product

Atlas Training Series Product

Procedure

  1. In Data operator definition, set the dynamic dimension in the shape to -1. 
    1
2
3
4
5
    		 
        auto shape_data = vector<int64_t>({ 1,-1,-1 });
    TensorDesc desc_data(ge::Shape(shape_data), FORMAT_ND, DT_FLOAT);
    auto data = op::Data("data");
    data.update_input_desc_data(desc_data);
    data.update_output_desc_out(desc_data);
  2. At model build time, set INPUT_SHAPE and INPUT_FORMAT in the options argument passed to the aclgrphBuildModel call and specify the dynamic dimension size profiles by setting DYNAMIC_DIMS.
    • INPUT_FORMAT must be consistent with the format of each Data operator, and only ND is supported. Otherwise, model build fails.
    • INPUT_SHAPE is optional. If this parameter is not set, the shape of the corresponding Data nodes is used by default. Otherwise, the passed argument is used and updated to those of the corresponding Data nodes.
    1
2
3
4
5
6
7
    		 
    void PrepareOptions(std::map<std::string, std::string>& options) {
    options.insert({
        {ge::ir_option::INPUT_FORMAT, "ND"},             
        {ge::ir_option::INPUT_SHAPE, "data:1,-1,-1"},   
        {ge::ir_option::DYNAMIC_DIMS, "1,2;3,4;5,6;7,8"}  // At model build time, set the shape of Data operator as "1,1,2; 1,3,4; 1,5,6; 1,7,8".
    });
}

Precautions

  • The dynamic dimension size feature is exclusive with dynamic batch size, dynamic image size, and AI Pre-Processing (AIPP). Only one of them can be enabled at a time.
  • The format is "dim1,dim2,dim3;dim4,dim5,dim6;dim7,dim8,dim9". Enclose all profiles in double quotation marks ("") and separate each profile with semicolons (;). The dim values match the –1 dimensions in the INPUT_SHAPE argument, with ordering preserved. For example: 
    1
2
3
4
5
6
7
    		 
    void PrepareOptions(std::map<std::string, std::string>& options) {
    options.insert({
        {ge::ir_option::INPUT_FORMAT, "ND"},             
        {ge::ir_option::INPUT_SHAPE, "data:1,1,40,-1;label:1,-1;mask:-1,-1"},   
        {ge::ir_option::DYNAMIC_DIMS, "20,20,1,1;40,40,2,2;80,60,4,4"}
    });
}

    The supported input shapes in model building are as follows:

    Profile 0: data(1,1,40,20)+label(1,20)+mask(1,1)

    Profile 1: data(1,1,40,40)+label(1,40)+mask(2,2)

    Profile 2: data(1,1,40,80)+label(1,60)+mask(4,4)

  • If this option is used to set the dynamic dimensions during model build, you need to perform the following operations before calling the model execution APIs to run an application project for inference:
    • Use the aclmdlSetInputDynamicDims API provided by AscendCL to set the runtime dimensions.
    • If aclmdlSetInputDynamicDims is not called, the maximum value within the dynamic dimension range is assigned by default during model execution.

    For details about the APIs, see " aclmdlSetInputDynamicDims".