Instructions
Switching Cards
MindStudio Insight allows you to switch Rank ID to view the memory information of different cards. Click the Rank ID text box in the upper part of the page and select the card number to be viewed from the drop-down list. After the switching, the operator memory curve and memory allocation/release details table of the corresponding card are displayed, as shown in Figure 1.
Switching the Display Dimension
MindStudio Insight allows you to switch Group By to view the operator memory curve in different dimensions. Click the Group By text box in the upper part of the page and select the dimension to be viewed from the drop-down list. This function is supported only in the dynamic curve scenario.
- Global: collects statistics on the memory size of operators in the reserved, allocated, and held states, and the total reserved memory size of PyTorch, as shown in Figure 2.
- Stream: collects statistics on the operator memory size in the reserved, allocated, and held states by stream, as shown in Figure 3.
- Component: collects statistics on the memory size of PyTorch operators in the reserved, allocated, and held states, and the memory size of GE in the reserved, allocated, and held states, as shown in Figure 4.
Zooming In and Out on a Memory Line Chart
MindStudio Insight allows you to left-click to drag select and zoom in on the selected part of the curve and right-click to zoom out on the curve. To improve the display performance, most points are hidden in the curve when the data volume is large. You can select a fine area to display all points or right-click the selected part to restore the original display effect.
In the curve, click and drag the mouse to the end point to be zoomed in and release the mouse. The selected area is zoomed in. If some points are still hidden, repeat the zoom-in operation to display the hidden points. Figure 5 shows the selected zoom-in area. Figure 6 shows the zoom-in effect.
- Click
in the upper right corner of the curve. If the button is dimmed, the curve is locked and cannot be zoomed in by clicking and dragging the mouse. You can click the button again or right-click the curve to restore the chart. The zoom-in function is enabled by default. - Click
in the upper right corner of the curve. The curve is restored to the initial state.
Searching for Operators
MindStudio Insight allows you to search for operators. The filter criteria column and control buttons are set in the memory allocation/release details table. You can set search criteria to display the operator details table. The search criteria include the operator name and the range of the memory size occupied by the operator (minimum value and maximum value). By default, the minimum and maximum memory sizes occupied by the operator in the selected card are displayed. You can adjust them according to the actual situation.
You can click Query to query the information, or click Reset to reset the search criteria and query the information again.
In the MindSpore static curve scenario, the memory allocation/release details table of operators in the static curve is displayed. In other scenarios, the memory allocation/release details table of operators in the dynamic curve is displayed.
- The dynamic curve search result is shown in Figure 7. You can search for operators whose names contain aten and memory size is 0–65 KB. Table 1 describes the fields in the table.If you select Only show allocated or released within the selected interval in the search row, the operators are those allocated or released within the selected interval in the curve.
Table 1 Dynamic curve fields Field
Description
Name
Operator name.
Size(KB)
Size of the memory to be allocated, in KB.
Allocation Time(ms)
Tensor memory allocation time, which is calculated from the time when the collection starts, in milliseconds.
Release Time(ms)
Tensor memory release time, which is calculated from the time when the collection starts, in milliseconds.
Duration(ms)
Memory holding time.
Active Release Time(ms)
Time when the memory is actually returned to the memory pool.
Active Duration(ms)
Actual memory usage duration.
Allocation Total Allocated(MB)
Total memory allocated by PyTorch and GE during operator memory allocation.
Allocation Total Reserved(MB)
Total memory reserved by PyTorch and GE during operator memory allocation.
Allocation Total Active(MB)
Total memory allocated to the current stream during operator memory allocation (including the memory that is reused by other streams but is not released).
Release Total Allocated(MB)
Size of the memory used by PyTorch and GE in the memory pool after the operator memory is released.
Release Total Reserved(MB)
Size of the memory occupied by PyTorch and GE in the memory pool after the operator memory is released.
Release Total Active(MB)
Total memory reused by other streams in the PyTorch and GE memory after the operator memory is released.
Stream
Memory address of an AscendCL stream, which is used to mark different AscendCL streams.
Operation
Click Show in Timeline to view the corresponding operator on the timeline page.
- In static curve mode, the search function is shown in Figure 8. You can search for operators whose names contain data and memory size is 0–35 KB. Table 2 describes the fields in the table.
Table 2 Static curve fields Field
Description
Device ID
Sequence number of the device that allocates memory. The default value is host.
Name
Operator name.
Node Index Start
Search start node.
Node Index End
Search end node.
The value 4294967295 is the final end node of the index. In the static curve, the value of this node is the actual end node plus 1. For example, if the actual end node is 32, the index of the node in the horizontal coordinate in the static curve is 33, and the end value of the node index in the table is 4294967295.
Size(MB)
Size of the memory to be allocated, in MB.
Operation
Click Show in Timeline to view the corresponding operator on the timeline page.
- When Group By is set to Component, the operator search function is not supported. Figure 9 shows the page. Table 3 describes the fields.
Table 3 Fields Field
Description
Component
Component name.
Peak Memory Reserved(MB)
Peak size of the memory occupied by a component, in MB. Only components whose peak memory usage is greater than or equal to 100 MB are displayed.
Timestamp(ms)
Time when the memory usage of a component reaches the peak value, in milliseconds.
Highlighting
When Group By is set to Overall and you move the pointer to a data record in the table (the curve is zoomed in to display all operators in the table), if the point (including the allocation time and release time) corresponding to the data record is displayed above the curve, the corresponding point on the curve is highlighted. This helps you quickly locate the operator.
When Group By is set to Component, highlighting is not supported.
Move the pointer to the red box in the table. The operator position is highlighted in the curve, as shown in Figure 10.
Comparing the Performance of Cards
MindStudio Insight allows you to compare the memory performance of cards. For details about how to set the comparison data, see Data Comparison.
In card performance comparison mode, the value of Rank ID is fixed and cannot be switched. Group By can be set only to Global or Component. The operator memory curve displays the memory changes of two cards, so that you can view the memory comparison trend between the two cards. The memory allocation/release details table displays the data difference between two cards. You can search for operators by name and memory size. The minimum memory size can be a negative value, and the search criteria are limited to the comparison result, as shown in Figure 11.
When Group By is set to Component, operators cannot be searched in the memory allocation/release details table.
You can click See more in the Details column of the memory allocation/release details table to view the baseline data and comparison data details.
