Memory Access
Memory access bound occurs when the memory of the vector or cube operator is moved in or out.
- Generally, you can calculate the memory access rate based on the data transfer volume and time consumed by each operator in the profiling file to determine whether memory access bound exists.
As shown in the preceding figure, the input data volume is 20000/48 x 2 x 8 = 6.5 KB. 48 indicates that data is allocated to 48 cores, 2 indicates that there are two 20000 shapes, 8 indicates int64, and the mte2 time is 1051 μs. Therefore, the bandwidth is 0.005 GB/s. This bandwidth is far from the theoretical bandwidth, which is definitely abnormal.
- Check whether the value of the aiv_mte2_ratio field in the profiling file exceeds 0.8. If a large number of operators exceed 0.8, optimization is required, as shown in the following figure.
Analysis Orientation
- Check whether the time consumption is proper.
- Vector operator fusion can effectively reduce the amount of data to be copied, thereby reducing the memory access pressure.
- Check whether the amount of data to be transferred is not 32-byte aligned. If the data is not aligned, the bandwidth is low. In this case, special processing needs to be performed in the operator.
Parent topic: Different Bottlenecks