AutoFuse Enabling Method

--enable_autofuse

Controls whether to enable the overall automatic fusion function.

Value:

• true: enables the function.
• false (default): disables the function.

If this option is not specified, the overall automatic fusion function is disabled. In this case, all the following control points become invalid, regardless of whether they have been set.

Configuration example:

--enable_autofuse=true

Constraints:

After the function is enabled, the automatic fusion of the Elemwise and Broadcast operators is enabled.

--autofuse_disable_pass

Controls whether to disable the extended fusion capability.

Value:

• reduce: disables the fusion capability for Reduce operators.
• concat: disables the fusion capability for Concat operators.

Use commas (,) to separate multiple values. By default, this option is left blank. The fusion capability is disabled for Reduce and Concat operators by default.

Configuration example:

--autofuse_disable_pass=reduce,concat

Restrictions:

The value cannot be the same as that of --autofuse_enable_pass.

--autofuse_enable_pass

Controls whether to enable the extended fusion capability.

Value:

• reduce: enables the fusion capability for Reduce operators.
• concat: enables the fusion capability for Concat operators.

Use commas (,) to separate multiple values. By default, this option is left blank. The fusion capability is disabled for Reduce and Concat operators by default.

Configuration example:

--autofuse_enable_pass=reduce,concat

Restrictions:

The value cannot be the same as that of --autofuse_disable_pass.

--autofuse_enhance_precision_blacklist

Controls whether to improve the precision on partial fusion nodes during automatic fusion.

The precision of an AscGraph will not be improved only when all AscIRs in the AscGraph are added to the blacklist.

Value:

A string of a combination of AscIR characters. Multiple types are separated by commas (,). The default value is an empty string, indicating that the precision of all AscGraphs is improved.

The AscIR type in this option is the value of type of the corresponding node in the dump graph. For example, if the value of type of a node in the dump graph is ge:Add, the value Add is configured here.

Configuration example:

--autofuse_enhance_precision_blacklist=Le,Where,Sub,Add,Sigmoid

Restrictions:

• The Sum, Mean, and Prod operators do not support low-precision types. By default, the precision must be improved. Therefore, they cannot be added to the precision improvement blacklist.
• Not improving the precision can achieve higher performance, but may cause precision issues. Therefore, after configuring not to improve the precision, ensure that the precision meets service requirements.
• For details about how to dump graphs, see Environment Variables > Graph Building > DUMP_GE_GRAPH.

--recomputation_threshold

Controls the threshold for automatic fusion recomputation.

It is used to define the reference threshold of a single-output operator. When the number of references of a single-output node exceeds the threshold, the first fusion is truncated on the current node.

Value: an integer ranging from 0 to 255. The default value is 1.

Configuration example:

--recomputation_threshold=5

Restrictions:

None

--experimental_enable_jit_executor_v2

Controls whether to enable graph partitioning and build.

Value:

• true: enables the function.
• false (default): disables the function.

Through graph partitioning and build, the original graph is split into N graphs at the boundary where symbols cannot be inferred. The output of the upstream graph is used as the input hint of the downstream graph for subsequent symbol inference and execution.

--experimental_enable_jit_executor_v2=true

Restrictions:

Graph partitioning is not supported in the following scenarios:

• Dynamic profile scenarios
• Graphs that contain resource operators (the input or input type is DT_RESOURCE, for example, TensorArrayWrite)
• Graphs that contain V1 control operators (such as "Switch", "StreamSwitch", "Merge", "StreamMerge", "Enter", "Exit", "LoopCond", and "NextIteration")
• Data preprocessing offload enabled
• AOE tuning enabled

--autofuse_enable_pgo

Controls whether to enable profile-guided optimization (PGO).

Through pre-board sampling, PGO picks the better-performing tiling solution to speed up model execution.

Value:

• true: enables the function.
• false (default): disables the function.

Configuration example:

--autofuse_enable_pgo=true

Restrictions:

Only static graphs can be tuned. If you configure this option for the first time, do not enable this function together with other profiling functions. There are no restrictions on subsequent configurations.

--codegen_compile_debug

Controls whether to retain the intermediate files generated during fused operator generation.

Value:

• true: retains the files.
• false (default): does not retain the files.

Configuration example:

--codegen_compile_debug=true

If this key is set to true, the kernel_meta_* folder is generated in the script execution directory, including the generated kernel source code, tiling source code, CMake project, and compilation result.

Constraints: none

--autofuse_att_algorithm

Controls the auto tiling algorithm.

Value:

• HighPerf: The high-performance algorithm is selected.
• AxesReorder: (default) The axis sorting algorithm is selected.

Configuration example:

--autofuse_att_algorithm=HighPerf

Restrictions:

• The axis sorting algorithm is the default algorithm. As an experimental approach, the high-performance algorithm does not guarantee superior execution performance consistently, despite its higher theoretical upper limit.
• If an invalid value is configured, the default value is used.

--att_accuracy_level

Controls the accuracy of the auto tiling algorithm. Theoretically, the higher the accuracy, the better the kernel performance.

Value:

• 1 (default): high-accuracy solution
• 0: low-accuracy solution

By default, the low-accuracy solution is used to ensure that the tiling time is not extended.

Configuration example:

--att_accuracy_level=1

Restrictions:

1. A high-accuracy solution may result in a better tiling result, but requires a longer tiling execution time. It is reverse for a low-precision solution.
2. If an invalid value is configured, the default value is used.

--att_enable_multicore_ub_tradeoff

Controls whether to enable the att_corenum_threshold and att_ub_threshold functions.

Value:

• true: enables the functions.
• false (default): disables the functions.

Configuration example:

--att_enable_multicore_ub_tradeoff=true

Restrictions: If an invalid value is configured, the default value is used.

--att_ub_threshold

Controls the tiling strategies for Auto Tiling to ensure that the tiling solution result is combined with the operator implementation. The UB usage is not lower than the value of this control point. If the UB usage does not meet the specified threshold, the maximum UB usage that can be solved is used. This control point helps locate performance-related problems.

Value: an integer ranging from 0 to 100. The default value is 20.

Configuration example:

--att_ub_threshold=20

Restrictions:

1. This option must be used together with --att_enable_multicore_ub_tradeoff. The --att_ub_threshold control point takes effect only when --att_enable_multicore_ub_tradeoff is enabled.
2. If an invalid value is configured, the default value is used.

--att_corenum_threshold

Controls the tiling strategies for Auto Tiling to ensure that the tiling solution result is combined with the operator implementation. The multi-core usage is not lower than the value set for this control point. If the multi-core usage does not meet the specified threshold, the maximum multi-core usage that can be solved is used. This control point helps locate performance-related problems.

Value: an integer ranging from 0 to 100. The default value is 40.

When --att_enable_multicore_ub_tradeoff is enabled, the default value is 40. Otherwise, no strategy is set.

Configuration example:

--att_corenum_threshold=40

Restrictions:

1. This option must be used together with --att_enable_multicore_ub_tradeoff. The --att_corenum_threshold control point takes effect only when --att_enable_multicore_ub_tradeoff is enabled.
2. If an invalid value is configured, the default value is used.

--att_profiling

Controls whether to enable profiling for Auto Tiling.

Value:

• true: enables profiling.
• false (default): disables profiling.

Configuration example:

--att_profiling=true

Restrictions:

1. This control point is used only to locate execution time-relevant problems of the Auto Tiling module.
2. If an invalid value is configured, the default value is used.

--disable_lifting

Controls whether to disable the lifting function.

Automatic fusion is used to fuse operators into a new operator named AscBackend, while lifting is used to roll back the AscBackend operator that does not meet the requirements to the original operators.

Value:

• true: disables the lifting function.
• false (default): enables the lifting function.

Configuration example:

--disable_lifting=true

Restrictions:

This control point is used only to locate the analysis problems of the Ascbackend rollback structure. Enabling this control point may cause the ApplyAdamD operator precision to be abnormal.

--autofuse_pgo_algo

Controls the PGO algorithm. The tiling method varies depending on the algorithm.

Value:

• core_select: (default) The core control algorithm is used.
• pruning: The pruning algorithm is used.

By default, the core control algorithm is used to ensure that the tiling time is not too long. Theoretically, the pruning algorithm may obtain a better tiling solution, but the tiling takes much longer.

Configuration example:

--autofuse_enable_pgo=true --autofuse_pgo_algo=core_select

Restrictions:

1. This option must be used together with --autofuse_enable_pgo. The --autofuse_pgo_algo control point takes effect only when --autofuse_enable_pgo is enabled.
2. If an invalid value is configured, the default value is used.

--autofuse_pgo_step_max

Controls the step of the PGO pruning algorithm.

Value: a power of 2 ranging from 2 to 1024. The default value is 16.

Configuration example:

--autofuse_enable_pgo=true --autofuse_pgo_algo=pruning --autofuse_pgo_step_max=16

Restrictions:

1. This option must be used together with --autofuse_pgo_algo. The --autofuse_pgo_step_max control point takes effect only when --autofuse_pgo_algo is set to pruning and the setting takes effect.

A smaller step value may result in a better tiling solution, but requires a longer tiling execution time. It is reverse for a larger step value.

2. If an invalid value is configured, the default value is used.

--autofuse_pgo_topn

Controls the number of solutions involved in PGO static tuning.

Value:

• 0: All solutions are used for static tuning.
• Any positive integer: indicates the number of solutions that can be selected. The default value is 5.

Configuration example:

--autofuse_enable_pgo=true --autofuse_pgo_topn=5

Restrictions:

1. This option must be used together with --autofuse_enable_pgo. The --autofuse_pgo_topn control point takes effect only when --autofuse_enable_pgo is enabled.
2. If an invalid value is configured, the default value is used.