--op_tune_mode

Applicability

Description

In certain foundation model scenarios, replacing some binary kernels with static kernels can achieve better performance. Based on this background, AOE adds this option to support static kernel tuning. It generates tuned kernels based on the input operator information .json file and saves them to a specified directory.

Currently, this function supports the following models:

Atlas A2 training product/Atlas A2 inference product

Atlas A3 training product/Atlas A3 inference product

See Also

If this option is set to static_kernel or fast, it must be used in conjunction with --op_tune_file and --precision_mode (= must_keep_origin_dtype).

Arguments

The arguments are as follows:

• static_kernel: static kernel tuning mode. In this mode, static kernels are used for compilation, and tuned kernels are generated, which are saved to the specified path.
• fast: compilation mode only. In this mode, compiled kernels are generated and saved to the specified path.

Examples

--op_tune_file=/home/lxl/json --op_tune_mode=static_kernel --precision_mode=must_keep_origin_dtype --op_tune_kernel_path=/home/lxl/kernel