Documentation

Converting Dump File Formats

Converting the Format of a Dump File

In the current version, dump files generated by running on the Ascend AI Processor can be converted into NumPy format.

This function is implemented through the msaccucmp.py script, which is stored in ${INSTALL_DIR}/tools/operator_cmp/compare. Replace ${INSTALL_DIR} with the actual CANN component directory. If the Ascend-CANN-Toolkit package is installed as the root user, the CANN component directory is /usr/local/Ascend/ascend-toolkit/latest..

The command is in the following format:

python3 msaccucmp.py convert -d dump_file [-out output] [-f format] [-s shape] [-o output_tensor] [-i input_tensor] [-c custom_script_path] [-v version] [-t type]

Table 1 describes the command-line options.

Table 1 Command-line options for format conversion

Option

Description

Required

-d

--dump_file

Dump file of a model running on the Ascend AI Processor.

You can specify a single file or a single path (recursive nesting is not supported, and only the parent directory of the file is supported). If you specify multiple files, separate the file names with commas (,), for example, -d /{PATH}/dump_file1,/{PATH}/dump_file2.

Yes

-out

--output

Directory of the converted data. Defaults to the current path.

You are not advised to configure directories that are different from those of the current user to avoid privilege escalation risks.

No

-f

--format
  • If the command contains -f, data is converted to a specified format. If the dump file contains the original_shape field, the data is tiled based on original_shape. For details about the supported format conversion types, see Supported Format Conversion Types.
  • If the command does not contain -f, the dump file is parsed.

No

-s

--shape

Shape to be set for FRACTAL_NZ conversion. The shape format is ([0-9]+,)+[0-9]+, where each number must be greater than 0. Valid only when -f is configured.

No

-o

--output_tensor

Index of the output to convert. Mutually exclusive with -i. Valid only when -f is configured.

If neither -o nor -i is included, all inputs and outputs are converted.

No

-i

--input_tensor

Index of the input to convert. Mutually exclusive with -o. Valid only when -f is configured.

No

-c

--custom_script_path

User-defined path to store the .py file for format conversion, which should be the parent directory of the format_convert directory. For details about the .py file requirements, see Preparing a Customized .py File for Format Conversion. Valid only when -f is configured.

You are advised not to invoke customized script files in directories of other users to avoid privilege escalation risks.

No

-v

--version

Dump file type. 1: data file after Protobuf serialization; 2 (default): data file in a custom format. Defaults to 2.

No

-t

--type

Type of an output file. Possible values are:

  • npy: The output file is saved in NumPy format.
  • msnpy: The output file is saved in NumPy format, which is generally used in MindSpore scenarios.
  • bin: The output file is saved in binary format.

The default value is npy.

No

Supported Format Conversion Types

The result is saved in the format of original_file_name.output.{index}.{shape}.npy or original_file_name.input.{index}.{shape}.npy, where shape is formatted as 1x3x224x224.

Currently, the following built-in format conversion types are supported:

  • FRACTAL_NZ to NCHW
  • FRACTAL_NZ to NHWC
  • FRACTAL_NZ to ND
  • HWCN to FRACTAL_Z
  • HWCN to NCHW
  • HWCN to NHWC
  • NC1HWC0 to HWCN
  • NC1HWC0 to NCHW
  • NC1HWC0 to NHWC
  • NCHW to FRACTAL_Z
  • NCHW to NHWC
  • NHWC to FRACTAL_Z
  • NHWC to HWCN
  • NHWC to NCHW
  • NDC1HWC0 to NCDHW

Generally, a non-4D format is converted from a 4D format. If a non-4D format can be converted into multiple formats, the non-4D format is valid only after it is converted back to the original 4D format. For example, NC1HWC0 can be converted into HWCN, NCHW, and NHWC. However, the NC1HWC0 data has only one type of 4D raw data (for example, HWCN), so NC1HWC0 can only be converted into HWCN. To identify the format type of raw data, see Advanced Features > Generating a Single-Operator Model in ATC Instructions.

Preparing a Customized .py File for Format Conversion

The user-defined .py file for format conversion can be used only for format conversion. The file security is ensured by users.

Prepare the file as follows:

  • The name foramt of the .py file is convert_{format_from}_to_{format_to}.py. The supported formats for format_from and format_to are as follows:
    • NCHW
    • NHWC
    • ND
    • NC1HWC0
    • FRACTAL_Z
    • NC1C0HWPAD
    • NHWC1C0
    • FSR_NCHW
    • FRACTAL_DECONV
    • C1HWNC0
    • FRACTAL_DECONV_TRANSPOSE
    • FRACTAL_DECONV_SP_STRIDE_TRANS
    • NC1HWC0_C04
    • FRACTAL_Z_C04
    • CHWN
    • DECONV_SP_STRIDE8_TRANS
    • NC1KHKWHWC0
    • BN_WEIGHT
    • FILTER_HWCK
    • HWCN
    • LOOKUP_LOOKUPS
    • LOOKUP_KEYS
    • LOOKUP_VALUE
    • LOOKUP_OUTPUT
    • LOOKUP_HITS
    • MD
    • NDHWC
    • C1HWNCoC0
    • FRACTAL_NZ
    • NCHDW
  • The content of the .py file is as follows:
    1
2
3
    def convert(shape_from, shape_to, array):
    
    return numpy_array
    Table 2 Parameters

    Parameter

    Description

    shape_from

    Shape of the one-dimensional array before conversion.

    shape_to

    (Optional) Shape of the one-dimensional array after conversion.

    array

    One-dimensional source data.

    return

    Return value, that is, NumPy array after conversion.
  • The directory of the .py file must meet the following requirement:

    The .py file must be stored in the format_convert directory. If the directory does not exist, create it.

Parent topic: Extended Functions