Comparing and Analyzing the Memory Degradation

The operator memory comparison result is displayed on the "MemoryCompare" and "MemoryCompareStatistic" sheets in the performance_comparison_result_*.xlsx.

  1. View the MemoryCompareStatistic sheet and find the top operators with the largest memory usage difference.
  2. View the MemoryCompare sheet, search for the top operators with the largest memory usage difference, and view the specific operators.

    For example, after the CANN and PTA software packages are upgraded onsite, OOM occurs. In this case, collect the Profiling results before and after the upgrade. According to the operator-level comparison, it is found that more than 10 GB memory is applied for aten::group_norm, as shown in Figure 1.

    Figure 1 Operator comparison