--op_tune_mode

Applicability

Description

In certain foundation model scenarios, replacing some binary kernels with static kernels can achieve better performance. Based on this background, AOE adds this option to support static kernel tuning. It generates tuned kernels based on the input operator information .json file and saves them to a specified directory.

Currently, this function supports the following models:

Atlas A2 training products/Atlas A2 inference products

Atlas A3 training products/Atlas A3 inference products

See Also

If this option is set to static_kernel or fast, it must be used in conjunction with --op_tune_file and --precision_mode (= must_keep_origin_dtype).

Arguments

The arguments are as follows:

• static_kernel: static kernel tuning mode. In this mode, static kernels are used for compilation, and tuned kernels are generated, which are saved to the specified path.
• fast: compilation mode only. In this mode, compiled kernels are generated and saved to the specified path.

Examples

--op_tune_file=/home/lxl/json --op_tune_mode=static_kernel --precision_mode=must_keep_origin_dtype --op_tune_kernel_path=/home/lxl/kernel