Performance Tuning Process

If the performance of the network ported to the AI processor for training is not satisfactory, you can perform the following steps to tune the performance.

Figure 1 Performance tuning process of TensorFlow network

1. If the performance is not satisfactory, you are advised to perform the following common operations to improve it:
   1. Enable the automatic mixed precision mode.
   2. Replace the GELU activation function.
   3. Use the AOE tool to tune subgraphs, operators, and gradient segmentation policies.
      

      The Atlas 350 Accelerator Card does not support the AOE tool.

   For details, see Basic Tuning.

2. Perform model training again and evaluate whether the training performance is satisfactory.
   • If the performance is satisfactory, the tuning is complete.
   • If the performance is not satisfactory, go to 3.
3. Use the Profiling tool to collect and analyze profile data.

   Refer to Profile Data Collection and Analysis to collect, parse, export, and analyze profile data.

4. Refer to Advanced Tuning to further improve the performance based on the identified performance bottleneck.
5. Perform model training again, conduct a regression test, and evaluate whether the training performance is satisfactory.
   • If the performance is satisfactory, the tuning is complete.
   • If the performance does not meet the requirements for the following products, perform the operations in Automatic AOE Tuning again.

     Atlas A3 training product / Atlas A3 inference product

     Atlas A2 training product / Atlas A2 inference product

     Atlas training product

   • If the performance does not meet the requirements for the Atlas 350 Accelerator Card, perform the operations in Profile Data Collection and Analysis again.