FormatForBinQuery

Function Usage

Sets the data formats to be queried by an operator binary file through Input and Output. The number of data types must be the same as that of Format or FormatList in one-to-one mapping.

During operator build, multiple .o files are generated and the operator binary files are indexed during running based on the data formats. Some operators support multiple data formats and are insensitive to data formats. In this case, you can use this API to map multiple data formats to the same operator binary file. Multiple data formats can reuse one .o file, reducing the number of binary files generated.

Assume that the input of an operator supports multiple data formats (ge::FORMAT_NC and ge::FORMAT_ND). If ge::FORMAT_NC input is used and the binary file of ge::FORMAT_ND can be reused without affecting the final result, the following configuration can be used:

this->Input("x")
            .ParamType(REQUIRED)
            .DataType({ge::DT_INT16, ge::DT_INT16})
            .Format({ge::FORMAT_NC, ge::FORMAT_ND})
            .FormatForBinQuery({ge::FORMAT_ND,ge::FORMAT_ND});

In this way, only one target file (.o) needs to be generated to support multiple data formats.

Prototype

OpParamDef &FormatForBinQuery(std::vector<ge::Format> formats);

Parameters

Returns

OpParamDef operator definition. For details, see OpParamDef.

Constraints

• The parameter length of FormatForBinQuery must be the same as that of Format or FormatList.
• This parameter cannot be used together with Scalar or ScalarList.
• This parameter cannot be used together with ValueDepend.
• After FormatForBinQuery is set, its data formats are used to replace the current Input and Output data formats. You can check whether a new combination exists before the replacement. If a new combination exists, it points to the corresponding binary file. If a new combination does not exist, FormatForBinQuery is invalid and the binary file is generated based on the original data formats. For details, see Example 1.

Example

• Example 1

          this->Input("x")
              .ParamType(REQUIRED)
              .DataType({ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16})
              .Format({ge::FORMAT_NC, ge::FORMAT_NCHW, ge::FORMAT_NHWC, ge::FORMAT_ND})
              .FormatForBinQuery({ge::FORMAT_NC, ge::FORMAT_NC, ge::FORMAT_ND, ge::FORMAT_NCHW});
          this->Output("y")
              .ParamType(REQUIRED)
              .DataType({ge::DT_FLOAT, ge::DT_FLOAT, ge::DT_FLOAT, ge::DT_FLOAT})
              .Format({ge::FORMAT_ND, ge::FORMAT_ND, ge::FORMAT_NC, ge::FORMAT_ND});

  As shown in the following figure, four binary files are generated before FormatForBinQuery is set. After FormatForBinQuery is set using the preceding code:

  • After the replacement, the fourth column uses the binary file of the second column, and the first and second columns use the binary file of the first column. The third column still uses its own binary file.
  • After the replacement, the first and second columns are completely the same so that the binary files are reused. The total operator binary files are reduced from four (bin1, bin2, bin3, and bin4) to three (bin1, bin2, and bin3).

  

• Example 2

          // Simple case: Two pairs are reused, that is, column 1 and column 2 -> column 1, and column 3 and column 4 -> column 4. Binary files 1 and 4 are generated in total. All supported formats are passed to the two binary files for running.
          this->Input("x")
              .ParamType(REQUIRED)
              .DataType({ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16})
              .Format({ge::FORMAT_NC, ge::FORMAT_NCHW, ge::FORMAT_NHWC, ge::FORMAT_ND})
              .FormatForBinQuery({ge::FORMAT_NC, ge::FORMAT_NC, ge::FORMAT_ND, ge::FORMAT_ND});
          this->Output("y")
              .ParamType(REQUIRED)
              .DataType({ge::DT_FLOAT, ge::DT_FLOAT, ge::DT_FLOAT, ge::DT_FLOAT})
              .Format({ge::FORMAT_ND, ge::FORMAT_ND, ge::FORMAT_ND, ge::FORMAT_ND});
  

• Example 3

          // Complex case: Multiple inputs/outputs can use FormatBinQuery at the same time. In this case, two pairs are reused, that is, columns 1 and 2 -> column 1; columns 3 and 4 -> column 3. Binary files 1 and 3 are generated in total. All supported formats are passed to the two binary files for running.
          this->Input("x")
              .ParamType(REQUIRED)
              .DataType({ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16})
              .Format({ge::FORMAT_NC, ge::FORMAT_NCHW, ge::FORMAT_NHWC, ge::FORMAT_ND})
              .FormatForBinQuery({ge::FORMAT_NC, ge::FORMAT_NC, ge::FORMAT_NHWC, ge::FORMAT_NHWC});
          this->Input("y")
              .ParamType(REQUIRED)
              .DataType({ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16})
              .Format({ge::FORMAT_NC, ge::FORMAT_NCHW, ge::FORMAT_NHWC, ge::FORMAT_ND})
              .FormatForBinQuery({ge::FORMAT_NC, ge::FORMAT_NC, ge::FORMAT_NHWC, ge::FORMAT_NHWC});
          this->Output("z")
              .ParamType(REQUIRED)
              .DataType({ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16, ge::DT_FLOAT16})
              .Format({ge::FORMAT_NC, ge::FORMAT_NCHW, ge::FORMAT_NHWC, ge::FORMAT_ND})
              .FormatForBinQuery({ge::FORMAT_NC, ge::FORMAT_NC, ge::FORMAT_NHWC, ge::FORMAT_NHWC});