Function: set_dump

dump_list

(Required) List of network-wide models for data dump.

• In the model inference scenario, to dump all operators, set this parameter as follows:

  "dump_list":[{}]

  When multiple models or specific operators need to be dumped, model_name and layer must be used together.

• In the single-operator calling scenario (including single-operator model execution and single-operator API execution), you are advised to set dump_list as follows:

  "dump_list":[{}]

model_name

Model name. The value of model_name of each model must be unique.

• To load a model from a file, enter the model file name without the name extension. You can also set this parameter to the value of the outermost name field in the .json file after ATC-based model conversion.
• To load a model from memory, set this parameter to the value of the name field in the .json file after ATC-based model conversion.

layer

Name of the operator to be dumped. When I/O performance is poor, full data dump may trigger execution timeouts due to excessive data volume. Full dump is therefore not recommended; specify target operators for dumping instead. The value can be the name of the operator after ATC model conversion or the name of the original operator before conversion.

• Configure the operator name in each line in the format. Use commas (,) to separate operators.
• You do not need to set model_name. In this case, the corresponding operators of all models are dumped by default. If model_name is set, the corresponding operators of the model are dumped.
• If the input of the specified operator involves the data operator, the data operator information is dumped. To dump the data operator, enter the downstream nodes of the data operator.
• To dump all operators of a model, the layer field does not need to be included.

optype_blacklist

Dump data blocklist. Input and output data of operators of the specified types in the blocklist will not be dumped, allowing users to control the volume of data to be dumped via this configuration.

This function takes effect only when model data is dumped and dump_level is set to op. It can also be used together with opname_blacklist.

Configuration example:

{
	"dump":{
		"dump_list":[     
			{                                                                                
				"model_name":"ResNet-50",
				"optype_blacklist":[
				    {
					  "name":"conv"
					  "pos":["input0", "input1"]
					} 
				] 
			}
		],  
		"dump_path":"/home/output",
                "dump_mode":"input",
	}  
}

The preceding example indicates that the dump operation is not performed on input0 and input1 of the conv operator. conv indicates the operator type.

optype_blacklist contains the name and pos fields. Pay attention to the following during configuration:

• name: operator type, which supports types of the operators generated after ATC model conversion. This filter rule does not take effect if left empty.
• pos: operator inputs or outputs. The value must be in the inputn or outputn format, where n stands for the input/output index. This filter rule does not take effect if left empty.
• A maximum of 100 filter entries can be configured in optype_blacklist.
• If model_name is specified, the configuration only applies to operators under the corresponding model. If model_name is not specified, the configuration applies to operators of all models.

opname_blacklist

Dump data blocklist. Input and output data of operators of the specified names in the blocklist will not be dumped, allowing users to control the volume of data to be dumped via this configuration.

This function takes effect only when model data is dumped and dump_level is set to op. It can also be used together with optype_blacklist.

Configuration example:

{
	"dump":{
		"dump_list":[     
			{                                                                                
				"model_name":"ResNet-50",
				"opname_blacklist":[
				    {
					  "name":"conv"
					  "pos":["input0", "input1"]
					} 
				] 
			}
		],  
		"dump_path":"/home/output",
                "dump_mode":"input",
	}  
}

The preceding example indicates that the dump operation is not performed on input0 and input1 of the conv operator. conv indicates the operator name.

opname_blacklist contains the name and pos fields. Pay attention to the following during configuration:

• name: operator name, which supports names of the operators generated after ATC model conversion. This filter rule does not take effect if left empty.
• pos: operator inputs or outputs. The value must be in the inputn or outputn format, where n stands for the input/output index. This filter rule does not take effect if left empty.
• A maximum of 100 filter entries can be configured in opname_blacklist.
• If model_name is specified, the configuration only applies to operators under the corresponding model. If model_name is not specified, the configuration applies to operators of all models.

opname_range

Dump data range. The dump operation is performed on the data within the closed interval from begin to end.

This function takes effect only when model data is dumped and dump_level is set to op.

Configuration example:

{
	"dump":{
		"dump_list":[
			{
				"model_name":"ResNet-50",
				"opname_range":[{"begin":"conv1", "end":"relu1" }, {"begin":"conv2", "end":"pool1"}]
			}
		],
		"dump_mode":"output",
        "dump_level": "op",
        "dump_path":"/home/output"
	}
}

The above example indicates that dump operations are performed on data within the closed interval from conv1 to relu1 and from conv2 to pool1, where conv1, relu1, conv2, and pool1 are operator names.

Note the following during configuration:

• model_name cannot be left empty.
• The parameters in begin and end indicate operator names, which can be set to the names of operators after ATC model conversion.
• begin and end cannot be empty and must be non-data operators. If the inputs of operators within the range defined by begin and end involve data operators, the data operator information will also be dumped simultaneously.

dump_path

(Required) Directory for storing dump data files in the operating environment. The directory must be created in advance and the running user configured during installation must have the read and write permissions on the directory.

dump_mode

Dump mode.

• input: dumps operator inputs only.
• output (default): dumps operator outputs only.
• all: dumps both operator inputs and outputs.

  Note: If this parameter is set to all, the input data of some operators, such as collective communication operators HcomAllGather and HcomAllReduce, will be modified during execution. Therefore, the system dumps the operator input before operator execution and dumps the operator output after operator execution. In this way, the dumped input and output data of the same operator is flushed to drives separately, and multiple dump files are generated. After parsing the dump files, you can determine whether the data is an input or output based on the file content.

dump_level

Dump data level. The options are as follows:

• op: dumps data at the operator level.
• kernel: dumps data at the kernel level.
• all (default): dumps both op and kernel level data.

If the default value is used, there are a large number of dump files, for example, dump files starting with aclnn. If you have requirements on the dump performance or the memory resources are limited, you can set this parameter to the op level to improve the dump performance and reduce the number of dump files.

Note: An operator represents a computational logic (such as addition, subtraction, multiplication, and division). A kernel is the implementation that executes this computational logic, and requires allocation of specific computing devices to complete computation.

dump_op_switch

Whether to enable dump data collection in the single-operator calling scenario (including single-operator model execution and single-operator API execution).

• on: enabled.
• off (default): disabled.

dump_step

Iterations to be dumped. This parameter is not required in the inference scenario.

If this parameter is not configured, dump data will be generated for all iterations by default, which may result in a large amount of data. You are advised to specify iterations as required.

Separate multiple iterations using vertical bars (|), for example, 0|5|10. You can also use hyphens (-) to specify the iteration range, for example, 0|3-5|10.

Configuration example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
	"dump":{
		"dump_list":[     
			...... 
		],  
		"dump_path":"/home/output",
                "dump_mode":"output",
		"dump_op_switch":"off",
                "dump_step": "0|3-5|10"
	}  
}

In training scenarios, if you specify iterations for dump data collection via the dump_step parameter in acl.json, and meanwhile configure the ge.exec.dumpStep parameter (which also controls dump collection iterations) in the GEInitialize API, the last configured parameter shall take precedence. For details about the GEInitialize API, see Graph Development.

dump_data

Type of the operator dump content. The options are as follows:

• tensor (default): dumps operator data.
• stats: dumps operator statistics. The result file is in .csv format and contains the operator name, input/output data type, maximum value, and minimum value.

Dumping full data typically generates large data volumes and consumes substantial time. You can first dump operator statistics, identify potentially abnormal operators based on these statistical results, and then dump only the data of those operators.

dump_stats

Type of statistical data to collect when dump_data is set to stats.

Only the Atlas A2 training product / Atlas A2 inference product supports this parameter.

Available values for this parameter are listed below. If no value is specified, Max, Min, Avg, Nan, Negative Inf, and Positive Inf data are collected by default.

• Max: maximum value in operator statistics.
• Min: minimum value in operator statistics.
• Avg: average value in operator statistics.
• Nan: undefined or unrepresentable values in operator statistics. This option applies only to the floating-point types half, bfloat, and float.
• Negative Inf: negative infinity values in operator statistics. This option applies only to the floating-point types half, bfloat, and float.
• Positive Inf: positive infinity values in operator statistics. This option applies only to the floating-point types half, bfloat, and float.
• L2norm: L2 norm value in operator statistics.

Configuration example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
    "dump":{
	"dump_list":[     
		...... 
	],  
        "dump_path":"/home/output",
        "dump_mode":"output",
        "dump_data":"stats",
        "dump_stats":["Max", "Min"]
    }
}