Scenarios with Common Performance Issues

During the process of porting an LLM from external devices to Ascend devices and training it on the Ascend device, performance issues may arise. Common performance issues often arise in the two major scenarios of out-of-the-box performance optimization and performance degradation over time.

  • Out-of-the-box performance optimization: This primarily refers to users noticing poor performance when using the model on a GPU platform and directly optimizing the performance.
  • Performance degradation over time: It generally refers to users encountering performance degradation issues during the training process due to unexpected factors. These issues require identifying the causes of the performance degradation and resolving them.
Figure 1 Scenario