Data in ascend_pytorch_profiler_{Rank_ID}.db
This is a table schema file. You are advised to use MindStudio Insight to view the file or use a database development tool such as Navicat Premium to open the file. The profile data summarized by the current .db file is as follows:
STRING_IDS
Mapping table that stores mapping between IDs and character strings.
There is no enable or disable option for this table. It records the mapping between strings and IDs used on CANN. Generally, the value starts accumulating from 0.
Field |
Type |
Index |
Meaning |
|---|---|---|---|
id |
INTEGER |
Primary key |
ID corresponding to the string. |
value |
TEXT |
- |
String content. |
PYTORCH_API
Stores the API data on the framework side. Currently, only torch_npu API data is included.
It is controlled by torch_npu.profiler.ProfilerActivity.CPU of the Ascend PyTorch Profiler API.
Field |
Type |
Meaning |
|---|---|---|
startNs |
INTEGER |
OP API start time, in ns. |
endNs |
INTEGER |
OP API end time, in ns. |
globalTid |
INTEGER |
Global TID of the API. High-order 32 bits: PID; low-order 32 bits: TID |
connectionId |
INTEGER |
Used to query the connection ID in the CONNECTION_IDS table. If the connection ID does not exist, this field is left empty. |
name |
INTEGER |
Name of the OP API, STRING_IDS(name). |
sequenceNumber |
INTEGER |
OP No. |
fwdThreadId |
INTEGER |
ID of the OP forward thread. |
inputDtypes |
INTEGER |
Input data type, STRING_IDS(inputDtypes). |
inputShapes |
INTEGER |
Input shape, STRING_IDS(inputShapes) |
callchainId |
INTEGER |
This field is used to query the call stack information in the PYTORCH_CALLCHAINS table. If no stack information is available, this field is left empty. |
type |
INTEGER |
Data type, which can be op, queue, mstx, or python_trace. The data type is stored in the enumeration table ENUM_API_TYPE. |
CONNECTION_IDS
Stores the associations between framework APIs or between framework APIs and CANN APIs.
It is controlled by torch_npu.profiler.ProfilerActivity.CPU of the Ascend PyTorch Profiler API.
Field |
Type |
Meaning |
|---|---|---|
id |
INTEGER |
Corresponds to the connectionId field in the PYTORCH_API table. |
connectionId |
INTEGER |
Association ID. Currently, the association can be including task_queue, fwd_bwd, or torch-cann-task. |
PYTORCH_CALLCHAINS
Stores stack information on the framework side.
It is controlled by the with_stack parameter of the Ascend PyTorch Profiler API.
Field |
Type |
Meaning |
|---|---|---|
id |
INTEGER |
Corresponds to the callchainId field in the PYTORCH_API table. |
stack |
INTEGER |
ID of the string content of the current stack in the STRING_IDS table. |
stackDepth |
INTEGER |
Depth of the current stack. |
MEMORY_RECORD
Stores device memory usage records on the framework side.
It is controlled by the profile_memory parameter of the Ascend PyTorch Profiler API.
Field |
Type |
Meaning |
|---|---|---|
component |
INTEGER |
ID of the component name (GE, PTA, or PTA+GE) in the STRING_IDS table. |
timestamp |
INTEGER |
Timestamp. |
totalAllocated |
INTEGER |
Total allocated memory. |
totalReserved |
INTEGER |
Total reserved memory. |
totalActive |
INTEGER |
Total memory allocated to the PTA flow. |
streamPtr |
INTEGER |
AscendCL stream address. |
deviceId |
INTEGER |
Device ID. |
OP_MEMORY
Stores operator memory usage information integrated based on MEMORY_RECORD on the framework side.
It is controlled by the profile_memory parameter of the Ascend PyTorch Profiler API.
Field |
Type |
Meaning |
|---|---|---|
name |
INTEGER |
torch and GE operator name, STRING_IDS(name). |
size |
INTEGER |
Size of the memory occupied by the operator, in bytes. |
allocationTime |
INTEGER |
Operator memory allocation time, in ns. |
releaseTime |
INTEGER |
Operator memory deallocation time, in ns. |
activeReleaseTime |
INTEGER |
Actual time when the memory is returned to the memory pool, in ns. |
duration |
INTEGER |
Memory occupation time, in ns. |
activeDuration |
INTEGER |
Actual memory occupation time, in ns. |
allocationTotalAllocated |
INTEGER |
Total memory allocated to PTA and GE during operator memory allocation, in bytes. |
allocationTotalReserved |
INTEGER |
Total memory occupied by PTA and GE during operator memory allocation, in bytes. |
allocationTotalActive |
INTEGER |
Total memory allocated for the current stream during operator memory allocation, in bytes. |
releaseTotalAllocated |
INTEGER |
Total memory allocated to PTA and GE during operator memory deallocation, in bytes. |
releaseTotalReserved |
INTEGER |
Total memory occupied by PTA and GE during operator memory deallocation, in bytes. |
releaseTotalActive |
INTEGER |
Total memory allocated for the current stream during operator memory deallocation, in bytes. |
streamPtr |
INTEGER |
AscendCL stream address. |
deviceId |
INTEGER |
Device ID. |
RANK_DEVICE_MAP
Stores mapping between rankId and deviceId.
There is no enable or disable option for this table. It is generated by default when the ascend_pytorch_profiler_{Rank_ID}.db file is exported.
Field |
Type |
Meaning |
|---|---|---|
rankId |
INTEGER |
Node ID in the cluster scenario. The value -1 indicates that rankId was not set. |
deviceId |
INTEGER |
Device ID on the node. The value -1 indicates that the device ID was not collected. |
STEP_TIME
Stores the start time of the step profiling.
It is controlled by the parameters of the torch_npu.profiler.schedule class of the Ascend PyTorch Profiler API.
Field |
Type |
Meaning |
|---|---|---|
id |
INTEGER |
Step ID. |
startNs |
INTEGER |
Step start time, in ns. |
endNs |
INTEGER |
Step end time, in ns. |
GC_RECORD
Stores GC events profiled by the profiler.
It is controlled by the gc_detect_threshold parameter of the Ascend PyTorch Profiler API.
Field |
Type |
Meaning |
|---|---|---|
startNs |
INTEGER |
Start time of a GC event, in ns. |
endNs |
INTEGER |
End time of a GC event, in ns. |
globalTid |
INTEGER |
Global TID of a GC event. |
ROCE
Stores the RoCE bandwidth data.
Control switch:
- --sys-io-profiling and --sys-io-sampling-freq of the msprof command
- sys_io of Ascend PyTorch Profiler
Field |
Type |
Meaning |
|---|---|---|
deviceId |
INTEGER |
Device ID. |
timestampNs |
INTEGER |
Local time, in ns. |
bandwidth |
INTEGER |
Bandwidth, in Byte/s. |
rxPacketRate |
NUMERIC |
Packet Rx rate, in packet/s. |
rxByteRate |
NUMERIC |
Rate of receiving bytes, in Byte/s. |
rxPackets |
INTEGER |
Total number of received packets, in packets. |
rxBytes |
INTEGER |
Total number of received bytes, in bytes. |
rxErrors |
INTEGER |
Total number of received error packets, in packets. |
rxDropped |
INTEGER |
Total number of lost received packets, in packets. |
txPacketRate |
NUMERIC |
Packet Tx rate, in packet/s |
txByteRate |
NUMERIC |
Rate of sending bytes, in Byte/s. |
txPackets |
INTEGER |
Total number of sent packets, in packets. |
txBytes |
INTEGER |
Total number of sent bytes. |
txErrors |
INTEGER |
Total number of sent error packets, in packets. |
txDropped |
INTEGER |
Total number of lost packets sent, in packets. |
funcId |
INTEGER |
Port. |
NIC
Stores NIC information over time.
Control switch:
- --sys-io-profiling and --sys-io-sampling-freq of the msprof command
- sys_io of Ascend PyTorch Profiler
Field |
Type |
Meaning |
|---|---|---|
deviceId |
INTEGER |
Device ID. |
timestampNs |
INTEGER |
Local time, in ns. |
bandwidth |
INTEGER |
Bandwidth, in Byte/s. |
rxPacketRate |
NUMERIC |
Packet Rx rate, in packet/s. |
rxByteRate |
NUMERIC |
Rate of receiving bytes, in Byte/s. |
rxPackets |
INTEGER |
Total number of received packets, in packets. |
rxBytes |
INTEGER |
Total number of received bytes, in bytes. |
rxErrors |
INTEGER |
Total number of received error packets, in packets. |
rxDropped |
INTEGER |
Total number of lost received packets, in packets. |
txPacketRate |
NUMERIC |
Packet Tx rate, in packet/s |
txByteRate |
NUMERIC |
Rate of sending bytes, in Byte/s. |
txPackets |
INTEGER |
Total number of sent packets, in packets. |
txBytes |
INTEGER |
Total number of sent bytes. |
txErrors |
INTEGER |
Total number of sent error packets, in packets. |
txDropped |
INTEGER |
Total number of lost packets sent, in packets. |
funcId |
INTEGER |
Port. |
HCCS
Stores the HCCS bandwidth data.
Control switch:
- --sys-interconnection-profiling and --sys-interconnection-freq of the msprof command
- sys_interconnection of Ascend PyTorch Profiler
Field |
Type |
Meaning |
|---|---|---|
deviceId |
INTEGER |
Device ID. |
timestampNs |
INTEGER |
Local time, in ns. |
txThroughput |
NUMERIC |
Tx bandwidth, in byte/s. |
rxThroughput |
NUMERIC |
Rx bandwidth, in byte/s. |
PCIE
Stores the PCIe bandwidth data.
Control switch:
- --sys-interconnection-profiling and --sys-interconnection-freq of the msprof command
- sys_interconnection of Ascend PyTorch Profiler
Field |
Type |
Meaning |
|---|---|---|
deviceId |
INTEGER |
Device ID. |
timestampNs |
INTEGER |
Local time, in ns. |
txPostMin |
NUMERIC |
Minimum bandwidth for sending PCIe Post data at the Tx side, in Byte/s. |
txPostMax |
NUMERIC |
Maximum bandwidth for sending PCIe Post data at the Tx side, in Byte/s. |
txPostAvg |
NUMERIC |
Average bandwidth for sending PCIe Post data at the Tx side, in Byte/s. |
txNonpostMin |
NUMERIC |
Minimum bandwidth for sending PCIe Non-Post data at the Tx side, in Byte/s. |
txNonpostMax |
NUMERIC |
Maximum bandwidth for sending PCIe Non-Post data at the Tx side, in Byte/s. |
txNonpostAvg |
NUMERIC |
Average bandwidth for sending PCIe Non-Post data at the Tx side, in Byte/s. |
txCplMin |
NUMERIC |
Minimum completion packet size for write requests at the Tx side, in Byte/s. |
txCplMax |
NUMERIC |
Maximum completion packet size for write requests at the Tx side, in Byte/s. |
txCplAvg |
NUMERIC |
Average completion packet size for write requests at the Tx side, in Byte/s. |
txNonpostLatencyMin |
NUMERIC |
Minimum transmission latency in PCIe Non-Post mode at the Tx side, in ns. |
txNonpostLatencyMax |
NUMERIC |
Maximum transmission latency in PCIe Non-Post mode at the Tx side, in ns. |
txNonpostLatencyAvg |
NUMERIC |
Average transmission latency in PCIe Non-Post mode at the Tx side, in ns. |
rxPostMin |
NUMERIC |
Minimum bandwidth for sending PCIe Post data at the Rx side, in Byte/s. |
rxPostMax |
NUMERIC |
Maximum bandwidth for sending PCIe Post data at the Rx side, in Byte/s. |
rxPostAvg |
NUMERIC |
Average bandwidth for sending PCIe Post data at the Rx side, in Byte/s. |
rxNonpostMin |
NUMERIC |
Minimum bandwidth for sending PCIe Non-Post data at the Rx side, in Byte/s. |
rxNonpostMax |
NUMERIC |
Maximum bandwidth for sending PCIe Non-Post data at the Rx side, in Byte/s. |
rxNonpostAvg |
NUMERIC |
Average bandwidth for sending PCIe Non-Post data at the Rx side, in Byte/s. |
rxCplMin |
NUMERIC |
Minimum completion packet size for write requests at the Rx side, in Byte/s. |
rxCplMax |
NUMERIC |
Maximum completion packet size for write requests at the Rx side, in Byte/s. |
rxCplAvg |
NUMERIC |
Average completion packet size for write requests at the Rx side, in Byte/s. |