Automatic AOE Tuning

The AOE tool continuously iterates tuning policies through a closed-loop feedback mechanism of policy generation, compilation, and verification in the operating environment, and finally obtains the optimal one. This helps fully utilize hardware resources and improve network performance. During model training, the AOE tool can be enabled to tune subgraphs, operators, and gradients. After the tuning is complete, the optimal tuning policy is added to the repository. When the model is trained again, you can directly use the repository for efficient tuning, without enabling the tuning function.

The AOE tuning feature supports only the following products:

Atlas A3 training product/Atlas A3 inference product
Atlas A2 training product/Atlas A2 inference product
Atlas training product

You are advised to use the AOE tool to perform tuning in the following sequence.

For the Atlas A2 training product/Atlas A2 inference product,Atlas A3 training product/Atlas A3 inference product, subgraph tuning is not supported.

You can enable AOE tuning in training scenarios using either of the following methods:

Set the environment variable.

 # 1: subgraph tuning; 2: operator tuning; 4: gradient tuning
export AOE_MODE=2

Add aoe_mode of AOE to the training script before initializing the NPU, to specify a tuning mode.

import npu_device as npu
npu.global_options().aoe_config.aoe_mode="2"
npu.open().as_default()

For details about the restrictions and functions of the AOE tool, see AOE Tuning Tool.

Parent topic: Basic Tuning