What Do I Do If Log Flush Fails?

Ascend EP Standard Form

If device-side system logs fail to be exported through the msnpureport tool:

Perform the following steps:

  1. Run the related command to view the printed information, for fault locating and problem solving.

    If the fault persists, go to 2.

  2. Run the following command on the host to check whether the drive space of the log storage path (path where the msnpureport tool is running) on the host is full:
    df -h
  3. If the fault persists: After obtaining the logs, click here to contact technical support.

If the flush of app logs fails:

Check whether the app logs (including the plog logs in the $HOME/ascend/log/ directory and the device-id logs) are flushed properly. If no, perform the following steps:

  1. Check whether there is any error log file in the host-side directory /var/log/messages.
    AArch64 architecture:
    cat /var/log/messages

    x86_64 architecture:

    cat /var/log/syslog

    If the fault persists, go to 2.

  2. Run the following command on the host to check whether the space of the log flush directory ($HOME/ascend/log/) is sufficient:
    df -h

    If the fault persists, go to 3.

  3. On the host, use the msnpureport tool to export device-side system logs and check whether there are any error logs.

    For details, see msnpureport Tool.

  4. If the fault persists: After obtaining the logs, click here to contact technical support.

Open Control CPU Form

If the flush of app logs fails:

Perform the following steps:

  1. Check whether the dynamic library on which the app process depends is correct:
    ldd xxx

    Replace xxx with the binary's process.

  2. Check whether the disk space of the log storage path /var/log/npu/slog is exhausted.
    df -h
  3. Check whether the slogd process exists.
    ps -elf | grep slogd

    If information about the slogd process is output, the slogd process exists.

  4. If none of the above abnormalities exist yet app logs still fail to be flushed to the disk, rectify the fault by referring to Restarting Log Processes.

If flush of the system logs fails:

Perform the following steps:

  1. Check whether the slogd and sklogd processes exist:
    ps -elf | grep log

    If the process information is output, the log process exists.

  2. Check whether the disk space of the log storage path /var/log/npu/slog is exhausted.
    df -h
  3. If none of the above abnormalities exist yet system logs still fail to be flushed to the disk, rectify the fault by referring to Restarting Log Processes.

Ascend RC Form

If the flush of app logs fails:

Perform the following steps:

  1. Check whether the dynamic library on which the app process depends is correct:
    ldd xxx

    Replace xxx with the binary app process.

  2. Check whether the disk space of the log storage path /var/log/npu/slog is exhausted.
    df -h
  3. Check whether the slogd process exists.
    ps -elf | grep slogd

    If information about the slogd process is output, the slogd process exists.

    For the Atlas 200I/500 A2 inference product, if the slogd process does not exist: After obtaining the logs, click here to contact technical support.

  4. If none of the above abnormalities exist yet app logs still fail to be flushed to the disk, rectify the fault by referring to Restarting Log Processes.

If flush of the system logs fails:

Perform the following steps:

  1. Check whether the related log processes (slogd and sklogd) exist.
    ps -elf | grep log

    If process information is displayed, the related log processes exist.

    For the Atlas 200I/500 A2 inference product, if the processes do not exist: After obtaining the logs, click here to contact technical support.

  2. Check whether the disk space of the log storage path /var/log/npu/slog is exhausted.
    df -h
  3. If none of the above abnormalities exist yet system logs still fail to be flushed to the disk, rectify the fault by referring to Restarting Log Processes.