Graph Build and Run

Overview

This feature is not supported by the Atlas 200I/500 A2 inference products.

Currently, only dynamic dimension configuration in the graph mode is supported during graph build and running: Use the Session parameter to set the profiles. The inputs can be fed using datasets or placeholders, or both. If both are used, only one type of the inputs can be dynamic.

Procedure

1. In Data operator definition, set the dynamic dimension in the shape to -1.

   1 2 3 4 5

   auto shape_data = vector<int64_t>({ 1,-1,-1 }); TensorDesc desc_data(ge::Shape(shape_data), FORMAT_ND, DT_FLOAT); auto data = op::Data("data"); data.update_input_desc_data(desc_data); data.update_output_desc_out(desc_data);

2. When building and running a graph, set the ge.inputShape/ge.dynamicNodeType information in options of the Session and AddGraph APIs, and specify the dimension using ge.dynamicDims.
   

   • ge.inputShape must be set.
   • When the dynamic dimension is configured in the graph mode, the input sequence of ge.inputShape set in the script must be in the same alphabetical order as the names of the data nodes. For example, assume that there are three inputs: label, data, and mask. The input sequence of ge.inputShape should be data, label, and mask.

     1	"data:1,1,40,-1;label:1,-1;mask:-1,-1"

   • In the scenario of building and running the graph that supports dynamic dimensions, if there are multiple inputs, the internal format can be specified for non-profiled inputs, but not for profiled inputs. For details, see Specifying the Internal Formats of Graph Inputs and Outputs.

   1 2 3 4 5 6 7

   std::map<std::string, std::string> options = {{"ge.inputShape", "data:1,1,40,-1;label:1,-1;mask:-1,-1"}, {"ge.dynamicDims", "20,20,1,1;40,40,2,2;80,60,4,4"}, {"ge.dynamicNodeType", "1"}}; // Add the dynamic dimension parameter to the session. ge::GeSession *session = new ge::GeSession(options); // Add the dynamic dimension parameter to the graph. session->AddGraph(graph_id, graph, options);

• ge.inputShape indicates the input shape. In this example, the network model has three inputs: data (1, 1, 40, -1), label (1, -1), and mask (-1, -1). -1 denotes a dynamic dimension, which can be set by using ge.dynamicDims.
• ge.dynamicDims is used to set the dynamic dimension profiles. These profiles are separated by semicolons (;). The dimension values match the -1 placeholders in the ge.inputShape argument with ordering preserved, and the number of -1 placeholders equals the number of dimension sizes of each profile. Based on the ge.inputShape information, if ge.dynamicDims is set to "20,20,1,1;40,40,2,2;80,60,4,4", the input shape supports three profiles. The input supports the following shape profiles at model build time:
  • Profile 0: data(1,1,40,20)+label(1,20)+mask(1,1)
  • Profile 1: data(1,1,40,40), label(1,40), mask(2,2)
  • Profile 2: data(1,1,40,80)+label(1,60)+mask(4,4)
• ge.dynamicNodeType sets the type of a dynamic input node. The value 0 indicates a dataset dynamic input and the value 1 indicates a placeholder dynamic input. Only one type of dynamic inputs is allowed, dataset or placeholder.