Detailed Description of Data Sources

Overview

This document provides a detailed description of the various data sources supported by Ascend-faultdiag-toolkit, including:

  • Data types and specific data items
  • Data collection methods (SSH online collection/offline log parsing)
  • Data source locations (specific commands/file paths)

The Ascend-faultdiag-toolkit supports data source collection from the following three device types:

  • Host
  • Baseboard Management Controller (BMC)
  • Switch

Each device supports the following two collection methods:

  • SSH online collection: Connects to the device via SSH and executes commands to obtain real-time data.
  • Offline log parsing: Parses exported log files to extract historical data.

Host Data Source

SSH Online Collection

Data ItemCollection CommandDescription
HostnamehostnameObtains the host name
NPU mapping informationnpu-smi info -mObtains the NPU mapping relationship (NPU ID, chip ID, physical ID)
Optical module informationhccn_tool -i {chip_phy_id} -optical -gObtains the optical module information of the specified physical chip
Link statushccn_tool -i {chip_phy_id} -link_stat -gObtains the link status of the specified physical chip
Statistics informationhccn_tool -i {chip_phy_id} -stat -gObtains the statistics information of the specified physical chip
LLDP informationhccn_tool -i {chip_phy_id} -lldp -gObtains the LLDP neighbor information of the specified physical chip
NPU typelspci | grep 'Device d80'Obtains the NPU device type
System serial numberdmidecode -s system-serial-numberObtains the system serial number
HCCS informationnpu-smi info -t hccs -i {npu_id} -c {chip_id}Obtains the HCCS information of the specified NPU and chip
SPOD informationnpu-smi info -t spod-info -i {npu_id} -c {chip_id}Obtains the SPOD information of the specified NPU and chip
MSNPUREPORT logmsnpureportGenerates and parses the MSNPUREPORT log
RoCE speedhccn_tool -i {chip_phy_id} -speed -gObtains the RoCE speed of the specified physical chip
RoCE duplex modehccn_tool -i {chip_phy_id} -duplex -gObtains the RoCE duplex mode of the specified physical chip
Network health statushccn_tool -i {chip_phy_id} -net_health -gObtains the network health status of the specified physical chip
Link statushccn_tool -i {chip_phy_id} -link -gObtains the link status of the specified physical chip
CDR informationhccn_tool -i {chip_phy_id} -scdr -t 5Obtains the CDR information of the specified physical chip
DFX configurationhccn_tool -i {chip_phy_id} -optical -g dfx_cfgObtains the DFX configuration of the specified physical chip
Optical module loopback testhccn_tool -i {npu_id} -optical -t {model}Performs an optical module loopback test

Offline Log Parsing

Host offline logs support three different configuration versions, each with its own set of file paths and parsing keywords.

Version 1 Configuration (ParseConfigCollectionV1)

File Structure

log_directory/
├── hccn_tool.log        # Network Configuration Tool Log
├── npu_card_info.log    # NPU Card Information Log
├── pcie_info.log        # PCIe Information Log
└── version_info.log     # Version Information Log

Data Items and Parsing Configuration

Data TypeParsing KeywordLog File PathDescription
LLDP"lldp"hccn_tool.logLink Layer Discovery Protocol information
SPEED"speed -g"hccn_tool.logPort speed information
OPTICAL"optical"hccn_tool.logOptical module information
LINK_STAT"link stat"hccn_tool.logLink statistics
STAT"stat"hccn_tool.logPerformance statistics
HCCN_LINK_STATUS"link"hccn_tool.logHCCN link status
CDR_SNR"cdr5 snr 1 times"hccn_tool.logCDR SNR information
SPOD_INFO"Collect spod-info info for all NPUs"npu_card_info.logSPOD information
HCCS"Collect hccs info for all NPUs"npu_card_info.logHCCS protocol information
NPU_TYPE"lspci"pcie_info.logNPU type information
SN"timeout 30s dmidecode -t1"version_info.logSerial number information

Version 2 Configuration (ParseConfigCollectionV2)

File Structure

log_directory/
├── hccn_log/
│   ├── net_conf.log     # Network Configuration Log
│   ├── optical.log      # Optical Module Log
│   └── stat.log         # Statistics Log
├── npu_smi_log/
│   └── npu_smi.log      # NPU SMI Log
└── pcie_log/
    └── pcie.log         # PCIe Log

Data Items and Parsing Configuration

Data TypeParsing KeyLog File PathDescription
LLDP"lldp"hccn_log/net_conf.logLink Layer Discovery Protocol information
SPEED"speed"hccn_log/net_conf.logPort speed information
OPTICAL"optical"hccn_log/optical.logOptical module information
LINK_STAT"link stat"hccn_log/optical.logLink statistics
NET_HEALTH"health info"hccn_log/optical.logNetwork health information
HCCN_LINK_STATUS"link info"hccn_log/optical.logHCCN link status
STAT"stat"hccn_log/stat.logPerformance statistics
SPOD_INFO"spod_info"npu_smi_log/npu_smi.logSPOD information
HCCS"hccs"npu_smi_log/npu_smi.logHCCS protocol information
NPU_TYPE"pcie"pcie_log/pcie.logNPU type information

Version 3 Configuration (ParseConfigCollectionV3)

File Structure

log_directory/
├── lldp.log        # LLDP Log
└── optical.log     # Optical Module Log

Data Items and Parsing Configuration

Data TypeParsing KeyLog File PathDescription
LLDP"lldp"lldp.logLink Layer Discovery Protocol information
SPEED"speed info"optical.logPort speed information
OPTICAL"optical"optical.logOptical module information
LINK_STAT"link stat"optical.logLink statistics
HCCN_LINK_STATUS"link info"optical.logHCCN link status

MSNPUREPORT Logs

In addition to the above configuration, parsing of logs generated by the MSNPUREPORT tool is also supported:

  • Path: Timestamp-named subdirectory under the log directory (e.g., 2023-10-01_14-30-00)
  • Content: Detailed NPU information

BMC Data Source

SSH Online Collection

Data Items and Sources

Data ItemCollection CommandDescription
BMC serial numberipmcget -d serialnumberObtains the BMC serial number
BMC date and timeipmcget -d timeObtains the current BMC time
SEL logipmcget -d sel -v listObtains the system event log
Sensor informationipmcget -t sensor -d listObtains the sensor list and status
Health eventipmcget -d healtheventsObtains the health event log
Diagnostic informationipmcget -d diaginfoObtains BMC diagnostic information and downloads it as a compressed package
Optical module historical log-Obtains optical module historical records

Diagnostic Information Compressed Package

  • Remote path: /tmp/dump_info.tar.gz
  • Local storage path: CommonPath.TOOL_HOME_BMC_DUMP_CACHE_DIR
  • Naming format: {host}_{sn_num}_{subfix_date_time}.tar.gz

Offline Log Parsing

Log Source

BMC offline logs are typically obtained through the diagnostic information archive (dump_info.tar.gz) generated by the ipmcget -d diaginfo command. After decompression, the directory structure is as follows:

dump_info/
└── AppDump/
    ├── bmc_network/
    │   └── network_info.txt      # Network information file
    ├── frudata/
    │   └── fruinfo.txt           # FRU information file
    ├── event/
    │   ├── sel.txt               # System event log
    │   └── current_event.txt     # Current health event
    ├── sensor/
    │   └── sensor_info.txt       # Sensor information
    ├── network_adapter/
    │   └── optical_module/
    │       └── optical_module_history_info_log.csv  # Optical module history log 1
    └── CpuMem/
        └── NpuIO/
            └── optical_module_history_info_log.csv  # Optical module history log 2

Data Items and Sources

Data ItemLog File PathParsing Method
BMC IP AddressAppDump/bmc_network/network_info.txtExtracts the "IP Address" field from the file
Serial NumberAppDump/frudata/fruinfo.txtExtracts "System Serial Number" from FRU information
SEL InformationAppDump/event/sel.txtDirectly reads SEL log content
Sensor InformationAppDump/sensor/sensor_info.txtDirectly reads sensor information content
Health EventAppDump/event/current_event.txtDirectly reads health event content
Optical Module Historical Log
  • AppDump/network_adapter/optical_module/optical_module_history_info_log.csv
  • AppDump/CpuMem/NpuIO/optical_module_history_info_log.csv
Parses CSV-formatted optical module history records to extract link interruption records

Switch Data Source

SSH Online Collection

Data ItemCollection CommandDescription
Switch Serial Numberdisplay license esnObtains the switch ESN serial number
Interface Summarydis int b | no-moreObtains the basic status of all interfaces
Switch Namedis cu | in sysnameObtains the switch system name
Optical Module Informationdis optical-module interface {interface}Obtains the optical module information of the specified interface
Bit Error Ratedisplay interface troubleshooting {interface}Obtains the troubleshooting information of the specified interface
LLDP Neighbordis lldp nei b | nObtains the LLDP neighbor summary
Active Alarmdisplay alarm active | no-moreObtains the current active alarms
Historical Alarmdisplay alarm history | no-moreObtains the historical alarm records
Interface Informationdisplay interface | no-moreObtains the detailed information of all interfaces
Current Timedisplay clock | include -Obtains the current time of the switch
HCCS Proxy Response Statisticsdisplay hccs proxy response statistics | no-moreObtains the HCCS proxy response statistics
HCCS Proxy Response Detailsdisplay hccs proxy response detail interface {interface}Obtains the HCCS proxy response details of the specified interface
HCCS Route Missdisplay hccs route miss statistics | no-moreObtains the HCCS route miss statistics
Port Link Statusdisplay for info enp s 1 c {chip_id} "get port link start 0 end 47" | no-moreObtains the chip port link status
Port Statisticsdisplay for info enp s 1 c {chip_id} "get port statistic count port {port_id} module {module} type 0 path 2" | no-moreObtains the port statistics
HCCS Port-Invalid Dropdisplay hccs port-invalid drop statistics | no-moreObtains the HCCS port-invalid drop statistics
Port Credit Back-Pressure Statisticsdisplay qos port-credit back-pressure statistics | no-moreObtains the port credit back-pressure statistics
HCCS Port SNRdisplay interface hilink snr | nObtains the HCCS port signal-to-noise ratio
Transceiver Informationdisplay interface transceiver verbose | no-moreObtains the detailed interface transceiver information
Interface Channel Informationdisplay interface information | no-moreObtains the interface channel information
Serdes Dump Informationdisplay for info enp s 1 c {chip_id} "get port serdes dump-info macro-id {port_id} lane-id {lane_id} hilink {type}" | no-moreObtains the Serdes dump information

Offline Log Parsing

Switch offline logs primarily come in the following two forms:

  • CLI command output log: Contains the execution results of various switch commands.
  • Diagnostic information output:Structured logs generated by switch diagnostic tools.

Log File Locations

  • CLI output log: Usually a single file (such as switch_cli_output.txt) or multiple files categorized by command.
  • Diagnostic information output: Usually a file with a diagnostic marker (such as diag_info.txt).

Data Types and Sources

Data TypeLog Source CharacteristicsDescription
Active Alarm DetailsContent block containing AlarmId, AlarmName, AlarmType, State : active fieldsExtracts detailed information of active alarms from CLI output logs
Historical Alarm DetailsContent block containing AlarmId, AlarmName, AlarmType, State : cleared fieldsExtracts detailed information of cleared alarms from CLI output logs
Active AlarmsTable containing Sequence, AlarmId, Severity, Date Time, Description fieldsExtracts active alarm summaries from CLI output logs
Historical AlarmsTable containing Sequence, AlarmId, Severity, Date Time, Description fields with historical markersExtracts historical alarm records from CLI output logs
LLDP NeighborsTable containing Local Interface, Exptime(s), Neighbor Interface, Neighbor Device fieldsExtracts LLDP neighbor information from CLI output logs
Optical Module InformationTable containing Items, Value, HighAlarm, HighWarn, LowAlarm, Status fieldsExtracts optical module monitoring data from CLI output logs
Interface SummaryTable containing Interface, PHY, Protocol, InUti, OutUti, inErrors, outErrors fieldsExtracts basic interface status from CLI output logs
Bit Error RateContent block containing Current state, Speed fieldsExtracts interface bit error rate information from CLI output logs
Interface InformationContent block containing current state, Description, Port Mode fieldsExtracts detailed interface configuration from CLI output logs
License ESNContent block containing MainBoard, ESN fieldsExtracts switch serial number from CLI output logs
System ClockContent block containing clock, Time Zone fieldsExtracts the current switch time from CLI output logs
Transceiver InformationContent block containing the transceiver information: markerExtracts detailed transceiver information from CLI output logs
HCCS Related InformationVarious tables and content blocks containing the HCCS keywordExtracts various statistical information related to the HCCS protocol from CLI output logs