怎么在不重启整机的情况下,重启NPU呢?
我使用npu-smi info ,是少了一张卡
运行
torch_npu._C._npu_init()
提示报错
[ERROR] 2025-03-29-15:28:30 (PID:568007, Device:0, RankID:-1) ERR00100 PTA call acl api failed [Error]: Failed to start the device. Rectify the fault based on the error information in the ascend log. E30005: [PID: 568007] 2025-03-29-15:28:30.057.950 The device failed to be connected. Possible Cause: 1. The device is broken. 2. The device failed to be started. Solution: 1.Restart the device. 2. Submit an issue on Gitee at https://gitee.com/ascend. TraceBack (most recent call last): Start aicpu executor failed, retCode=0x7020009 devId=0[FUNC:DeviceRetain][FILE:runtime.cc][LINE:3941] Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3690] Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3717] Check param failed, context can not be null.[FUNC:NewDevice][FILE:api_impl.cc][LINE:2457] New device failed, retCode=0x7010006[FUNC:SetDevice][FILE:api_impl.cc][LINE:2480] rtSetDevice execute failed, reason=[device retain error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53] open device 0 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161] Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3993] ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:5315] The argument is invalid.Reason: rtGetDevMsg execute failed, reason=[context pointer null]
怎么在不重启整机的情况下,重启NPU呢?
我使用npu-smi info ,是少了一张卡
运行
torch_npu._C._npu_init()
提示报错
[ERROR] 2025-03-29-15:28:30 (PID:568007, Device:0, RankID:-1) ERR00100 PTA call acl api failed [Error]: Failed to start the device. Rectify the fault based on the error information in the ascend log. E30005: [PID: 568007] 2025-03-29-15:28:30.057.950 The device failed to be connected. Possible Cause: 1. The device is broken. 2. The device failed to be started. Solution: 1.Restart the device. 2. Submit an issue on Gitee at https://gitee.com/ascend. TraceBack (most recent call last): Start aicpu executor failed, retCode=0x7020009 devId=0[FUNC:DeviceRetain][FILE:runtime.cc][LINE:3941] Check param failed, dev can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3690] Check param failed, ctx can not be NULL![FUNC:PrimaryContextRetain][FILE:runtime.cc][LINE:3717] Check param failed, context can not be null.[FUNC:NewDevice][FILE:api_impl.cc][LINE:2457] New device failed, retCode=0x7010006[FUNC:SetDevice][FILE:api_impl.cc][LINE:2480] rtSetDevice execute failed, reason=[device retain error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53] open device 0 failed, runtime result = 507033.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161] Check param failed, dev can not be NULL![FUNC:DeviceRetain][FILE:runtime.cc][LINE:3993] ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:5315] The argument is invalid.Reason: rtGetDevMsg execute failed, reason=[context pointer null]