How Do I Convert the Format of a Dump File?

Converting the Format of a Dump File

In the current version, dump files generated by running on the Ascend AI Processor can be converted into NumPy format.

This function is implemented by using the shape_conversion.py script, which is stored in the ${INSTALL_DIR}/tools/operator_cmp/compare. Replace ${INSTALL_DIR} with the actual CANN component directory. If the Ascend-CANN-Toolkit package is installed as the root user, the CANN component directory is /usr/local/Ascend/ascend-toolkit/latest.. The command format is as follows:

python3 shape_conversion.py -i dump_file_path -format format -o outputpath [-shape shape] [-tensor tensor] [-index index] [-custom custom_path]

The command-line options are described as follows:

• -i: dump file (including the path) of a model running on the Ascend AI Processor.
• -format: format of the converted file. For details about the supported format conversion types, see Supported Format Conversion Types.
• -o: directory of the converted file.

  You are not advised to configure directories that are different from those of the current user to avoid privilege escalation risks.

• -shape: (optional) shape to be set for FRACTAL_NZ conversion. The shape format is ([0-9]+,)+[0-9]+, where each number must be greater than 0.
• -tensor: (optional) whether the dump file to be converted is the input or output. Defaults to output.
• -index: (optional) sequence number of a tensor, indexed starting at 0.
• -custom: (optional) customized path to store the .py file for format conversion, which should be the parent directory of the format_convert directory. For details about the .py file requirements, see Preparing a Customized .py File for Format Conversion.

  You are advised not to invoke customized script files in directories of other users to avoid privilege escalation risks.

Supported Format Conversion Types

The result is saved in the format of original_file_name.output.{index}.{shape}.npy or original_file_name.input.{index}.{shape}.npy, where shape is formatted as 1x3x224x224.

If the custom format is the same as the built-in format, the custom format applies.

Currently, the following built-in format conversion types are supported:

• FRACTAL_NZ to NCHW
• FRACTAL_NZ to ND
• HWCN to FRACTAL_Z
• HWCN to NCHW
• HWCN to NHWC
• NC1HWC0 to HWCN
• NC1HWC0 to NCHW
• NC1HWC0 to NHWC
• NCHW to FRACTAL_Z
• NCHW to NHWC
• NHWC to FRACTAL_Z
• NHWC to HWCN
• NHWC to NCHW
• NDC1HWC0 to NCDHW

Generally, a non-4D format is converted from a 4D format. If a non-4D format can be converted into multiple formats, the non-4D format is valid only after it is converted back to the original 4D format. For example, NC1HWC0 can be converted into HWCN, NCHW, and NHWC. However, the NC1HWC0 data has only one type of 4D raw data (for example, HWCN), so NC1HWC0 can only be converted into HWCN. To identify the format type of raw data, see Advanced Features > Generating a Single-Operator Model in ATC Instructions.

Preparing a Customized .py File for Format Conversion

The user-defined .py file for format conversion can be used only for format conversion. The file security is ensured by users.

Prepare the file as follows:

• The name of the .py file is in convert_{format_from}_to_{format_to}.py format. The supported formats for format _from and format _to are as follows:
  • NCHW
  • NHWC
  • ND
  • NC1HWC0
  • FRACTAL_Z
  • NC1C0HWPAD
  • NHWC1C0
  • FSR_NCHW
  • FRACTAL_DECONV
  • C1HWNC0
  • FRACTAL_DECONV_TRANSPOSE
  • FRACTAL_DECONV_SP_STRIDE_TRANS
  • NC1HWC0_C04
  • FRACTAL_Z_C04
  • CHWN
  • DECONV_SP_STRIDE8_TRANS
  • NC1KHKWHWC0
  • BN_WEIGHT
  • FILTER_HWCK
  • HWCN
  • LOOKUP_LOOKUPS
  • LOOKUP_KEYS
  • LOOKUP_VALUE
  • LOOKUP_OUTPUT
  • LOOKUP_HITS
  • MD
  • NDHWC
  • C1HWNCoC0
  • FRACTAL_NZ
• The content of the .py file is as follows:

  1 2 3

  def convert(shape_from, shape_to, array): return numpy_array

  The parameters are described as follows:

  • shape_from: shape of the one-dimensional array before conversion
  • shape_to: (optional) shape of the one-dimensional array after conversion
  • array: one-dimensional source data
  • return: NumPy array after conversion
• The directory of the .py file must meet the following requirement:

  The .py file must be stored in the format_convert directory. If the directory does not exist, create it.