Automatic AOE Tuning

The AOE tool continuously iterates tuning policies through a closed-loop feedback mechanism of policy generation, compilation, and verification in the operating environment, and finally obtains the optimal one. This helps fully utilize hardware resources and improve network performance. During model training, the AOE tool can be enabled to tune subgraphs, operators, and gradients. After the tuning is complete, the optimal tuning policy is added to the repository. When the model is trained again, you can directly use the repository for efficient tuning, without enabling the tuning function.

The AOE tuning feature supports only the following products:

Atlas A3 training products/Atlas A3 inference products
Atlas A2 training products/Atlas A2 inference products
Atlas training products

You are advised to use the AOE tool to perform tuning in the following sequence:

Subgraph tuning is not supported for the Atlas A3 training products/Atlas A3 inference productsAtlas A2 training products/Atlas A2 inference products.

You can enable AOE tuning in training scenarios using either of the following methods:

Set the environment variable.

 # 1: subgraph tuning; 2: operator tuning; 4: gradient tuning
export AOE_MODE=2

Modify the training script and use aoe_mode to specify the tuning mode, for example:
- In sess.run mode, modify the training script as follows:
  1
  custom_op.parameter_map["aoe_mode"].s = tf.compat.as_bytes("2")
- In Estimator mode, modify the training script as follows:
  1 2 3
  config = NPURunConfig( session_config=session_config, aoe_mode=2)
- In keras mode, modify the training script as follows:
  1
  custom_op.parameter_map["aoe_mode"].s = tf.compat.as_bytes("2")

For details about the restrictions and functions of the AOE tool, see AOE Instructions.

Parent topic: Basic Tuning