310P3卡推理初始化报错
收藏回复举报
310P3卡推理初始化报错
t('forum.solved') 已解决
新人帖
发表于2025-01-10 11:25:38
0 查看
设备信息:

Ascend310P-driver-22.0.4.2-1.aarch64 

Ascend310P-firmware-1.84.15.2.220-1.noarch 

Version=22.0.4.2 

ascendhal_version=6.17.28 

aicpu_version=1.0 

tdt_version=1.0 

log_version=1.0 

prof_version=2.0 

dvppkernels_version=1.1 

tsfw_version=1.0 

Innerversion=V100R001C84SPC002B220 

package_version=6.0.0.2 

compatible_version=[V100R001C81,V100R001C84] 

 

NPU ID                         : 2 

Product Name                   : IT21PDDA011 

Model                          : NA 

Manufacturer                   : Huawei 

Serial Number                  : 2106030728ZEQ5004734 

Software Version               : 22.0.4.2 

Firmware Version               : 1.84.15.2.220 

Board ID                       : 0xab 

PCB ID                         : B 

BOM ID                         : 0 

PCIe Bus Info                  : 0000:02:00.0 

Slot ID                        : 2 

Class ID                       : NA 

PCI Vendor ID                  : 0x19E5 

PCI Device ID                  : 0xD500 

Subsystem Vendor ID            : 0x0200 

Subsystem Device ID            : 0x0100 

Chip Count                     : 1 

Chip Fault                     : 0 


CANN版本:6.0.1

torch_npu.npu.set_device(device_id) 或者 acl.rt.set_device(0) 设置设备时报告警:  
[WARNING] DRV(6206,python):2025-01-10-02:56:41.671.296 [ascend][curpid: 6206, 6274][drv][tsdrv][TsDrvLogicCqIrqWait 447]wait_logic_cq abnormal result show: err_code=16 devId=2 tsId=0 

[INFO] DRV(6206,python):2025-01-10-02:56:41.671.362 [ascend][curpid: 6206, 6274][drv][tsdrv][TsDrvLogicCqIrqWait 463]end, logic_cqid=0 reporCnt=0 devId=2 tsId=0 err_code=16. 

[DEBUG] RUNTIME(6206,python):2025-01-10-02:56:41.671.394 [npu_driver.cc:732] 6274 LogicCqReport: halCqReportIrqWait failed: device_id=1, ts_id=0, type=2, cq_id=0, drvRetCode=16 

[INFO] TDT(6206,python):2025-01-10-02:56:41.671.427 [log.cpp:47][ClientManager] GetInstance, deviceId[1],[client_manager.cpp:84:GetInstance]6274 

[INFO] TDT(6206,python):2025-01-10-02:56:41.671.447 [log.cpp:47][ClientManager] GetInstance, deviceId[1],[client_manager.cpp:84:GetInstance]6274 

[DEBUG] RUNTIME(6206,python):2025-01-10-02:56:41.671.493 [npu_driver.cc:243] 6274 RtGetRunMode: RunMode=1 [0:offline, 1:online] 

[INFO] DRV(6206,python):2025-01-10-02:56:41.671.530 [ascend][curpid: 6206, 6274][drv][tsdrv][TsDrvLogicCqIrqWait 428]start, logic_cqid=0 devId=2 tsId=0 


acl.mdl.load_from_file("/opt_model_bs1.om") 模型加载是会报错:
[EVENT] PROFILING(6029,python):2025-01-09-03:26:19.432.079 [msprof_callback_impl.cpp:199] >>> (tid:6029) Started to register profiling ctrl callback. 

[EVENT] PROFILING(6029,python):2025-01-09-03:27:03.385.195 [msprof_callback_impl.cpp:78] >>> (tid:6029) MsprofCtrlCallback called, type: 255 

[EVENT] PROFILING(6029,python):2025-01-09-03:27:03.385.266 [prof_acl_mgr.cpp:1190] >>> (tid:6029) Init profiling for dynamic profiling 

[ERROR] RUNTIME(6029,python):2025-01-09-03:27:24.883.763 [engine.cc:1263]6029 ReportExceptProc:Task exception! device_id=0, stream_id=3, task_id=1, type=83, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:27:24.885.652 [device_error_proc.cc:724]6029 CheckValid:The length of ringbuffer is invalid. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:27:24.885.666 [device_error_proc.cc:775]6029 ProcErrorInfo:Find error in ringbuffer. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:27:24.885.777 [stream.cc:1041]6029 GetError:Stream Synchronize failed, stream_id=3, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:27:24.885.788 [aicpu_err_msg.cc:182]6029 SendConfigMsgToAicpu:apicpu kernel launch failed, retCode=0x7150004. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:27:24.885.795 [aicpu_err_msg.cc:215]6029 SetErrMsgBufAddr:set err msg buf addr to aicpu failed, retCode=0x7150004, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.079.609 [engine.cc:1263]6029 ReportExceptProc:[LOAD][DEFAULT]Task exception! device_id=0, stream_id=5, task_id=4, type=75, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.080.163 [device_error_proc.cc:724]6029 CheckValid:[LOAD][DEFAULT]The length of ringbuffer is invalid. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.080.191 [device_error_proc.cc:775]6029 ProcErrorInfo:[LOAD][DEFAULT]Find error in ringbuffer. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.080.256 [engine.cc:1263]6029 ReportExceptProc:[LOAD][DEFAULT]Task exception! device_id=0, stream_id=5, task_id=124, type=20, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.080.530 [device_error_proc.cc:724]6029 CheckValid:[LOAD][DEFAULT]The length of ringbuffer is invalid. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.080.552 [device_error_proc.cc:775]6029 ProcErrorInfo:[LOAD][DEFAULT]Find error in ringbuffer. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.080.864 [engine.cc:1263]6029 ReportExceptProc:[LOAD][DEFAULT]Task exception! device_id=0, stream_id=5, task_id=244, type=12, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.123 [device_error_proc.cc:724]6029 CheckValid:[LOAD][DEFAULT]The length of ringbuffer is invalid. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.146 [device_error_proc.cc:775]6029 ProcErrorInfo:[LOAD][DEFAULT]Find error in ringbuffer. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.171 [task.cc:2914]6029 DoCompleteSuccess:[LOAD][DEFAULT]Model maintaince process error, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.188 [task.cc:2904]6029 PrintErrorInfo:[LOAD][DEFAULT]model maintaince execute failed device_id=0, stream_id=5, task_id=244, flip_num=0. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.204 [task.cc:2907]6029 PrintErrorInfo:[LOAD][DEFAULT]model_id=0, operation_type=5, stream_type=0, op_stream_id=5. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.408 [stream.cc:1041]6029 GetError:[LOAD][DEFAULT]Stream Synchronize failed, stream_id=5, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.419 [model.cc:481]6029 LoadCompleteByStream:[LOAD][DEFAULT]report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.426 [model.cc:481]6029 LoadCompleteByStream:[LOAD][DEFAULT]Failed to synchronize sink stream, retCode=0x7150004! 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.481 [context.cc:1684]6029 ModelLoadComplete:[LOAD][DEFAULT]report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.487 [context.cc:1684]6029 ModelLoadComplete:[LOAD][DEFAULT]Model load complete process failed, model_id=0, retCode=0x7150004. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.503 [logger.cc:851]6029 ModelLoadComplete:[LOAD][DEFAULT]Load model complete failed. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.536 [api_c.cc:2190]6029 rtModelLoadComplete:[LOAD][DEFAULT]ErrCode=507001, desc=[tsfw param illegal], InnerCode=0x7150004 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.542 [error_message_manage.cc:49]6029 FuncErrorReason:[LOAD][DEFAULT]report error module_type=3, module_name=EE8888 

[ERROR] RUNTIME(6029,python):2025-01-09-03:28:11.081.554 [error_message_manage.cc:49]6029 FuncErrorReason:[LOAD][DEFAULT]rtModelLoadComplete execute failed, reason=[tsfw param illegal] 

[ERROR] GE(6029,python):2025-01-09-03:28:11.081.612 [davinci_model.cc:554]6029 DoTaskSink: ErrorNo: 1343225859(Failed to call runtime API!) [LOAD][DEFAULT]Call rt api failed, ret: 0x7BC79 

[ERROR] GE(6029,python):2025-01-09-03:28:11.081.620 [davinci_model.cc:657]6029 Init: ErrorNo: 4294967295(failed) [LOAD][DEFAULT][Call][DoTaskSink] failed, model_id:1. 

[ERROR] GE(6029,python):2025-01-09-03:28:11.081.628 [model_manager.cc:1248]6029 LoadModelOffline: ErrorNo: 4294967295(failed) [LOAD][DEFAULT][Init][DavinciModel] failed, ret:507001. 

[ERROR] GE(6029,python):2025-01-09-03:28:11.099.619 [graph_loader.cc:141]6029 LoadModelFromData: ErrorNo: 507001() [LOAD][DEFAULT][Load][Model] failed, model_id:1. 

[ERROR] ASCENDCL(6029,python):2025-01-09-03:28:11.099.636 [model.cpp:274]6029 ModelLoadFromFileWithMem: [LOAD][DEFAULT][Model][FromData]load model from data failed, ge result[507001] 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.175.260 [engine.cc:1263]6029 ReportExceptProc:[LOAD][DEFAULT]Task exception! device_id=0, stream_id=5, task_id=4, type=75, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.176.978 [device_error_proc.cc:724]6029 CheckValid:[LOAD][DEFAULT]The length of ringbuffer is invalid. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.177.005 [device_error_proc.cc:775]6029 ProcErrorInfo:[LOAD][DEFAULT]Find error in ringbuffer. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.177.073 [engine.cc:1263]6029 ReportExceptProc:[LOAD][DEFAULT]Task exception! device_id=0, stream_id=5, task_id=124, type=20, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.178.434 [device_error_proc.cc:724]6029 CheckValid:[LOAD][DEFAULT]The length of ringbuffer is invalid. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.178.457 [device_error_proc.cc:775]6029 ProcErrorInfo:[LOAD][DEFAULT]Find error in ringbuffer. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.178.782 [engine.cc:1263]6029 ReportExceptProc:[LOAD][DEFAULT]Task exception! device_id=0, stream_id=5, task_id=244, type=12, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.128 [device_error_proc.cc:724]6029 CheckValid:[LOAD][DEFAULT]The length of ringbuffer is invalid. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.150 [device_error_proc.cc:775]6029 ProcErrorInfo:[LOAD][DEFAULT]Find error in ringbuffer. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.177 [task.cc:2914]6029 DoCompleteSuccess:[LOAD][DEFAULT]Model maintaince process error, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.194 [task.cc:2904]6029 PrintErrorInfo:[LOAD][DEFAULT]model maintaince execute failed device_id=0, stream_id=5, task_id=244, flip_num=0. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.210 [task.cc:2907]6029 PrintErrorInfo:[LOAD][DEFAULT]model_id=0, operation_type=5, stream_type=0, op_stream_id=5. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.437 [stream.cc:1041]6029 GetError:[LOAD][DEFAULT]Stream Synchronize failed, stream_id=5, retCode=0x4, [illegal param]. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.450 [model.cc:481]6029 LoadCompleteByStream:[LOAD][DEFAULT]report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.457 [model.cc:481]6029 LoadCompleteByStream:[LOAD][DEFAULT]Failed to synchronize sink stream, retCode=0x7150004! 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.505 [context.cc:1684]6029 ModelLoadComplete:[LOAD][DEFAULT]report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.512 [context.cc:1684]6029 ModelLoadComplete:[LOAD][DEFAULT]Model load complete process failed, model_id=0, retCode=0x7150004. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.525 [logger.cc:851]6029 ModelLoadComplete:[LOAD][DEFAULT]Load model complete failed. 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.565 [api_c.cc:2190]6029 rtModelLoadComplete:[LOAD][DEFAULT]ErrCode=507001, desc=[tsfw param illegal], InnerCode=0x7150004 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.572 [error_message_manage.cc:49]6029 FuncErrorReason:[LOAD][DEFAULT]report error module_type=3, module_name=EE8888 

[ERROR] RUNTIME(6029,python):2025-01-09-03:35:20.180.584 [error_message_manage.cc:49]6029 FuncErrorReason:[LOAD][DEFAULT]rtModelLoadComplete execute failed, reason=[tsfw param illegal] 

[ERROR] GE(6029,python):2025-01-09-03:35:20.180.611 [davinci_model.cc:554]6029 DoTaskSink: ErrorNo: 1343225859(Failed to call runtime API!) [LOAD][DEFAULT]Call rt api failed, ret: 0x7BC79 

[ERROR] GE(6029,python):2025-01-09-03:35:20.180.620 [davinci_model.cc:657]6029 Init: ErrorNo: 4294967295(failed) [LOAD][DEFAULT][Call][DoTaskSink] failed, model_id:2. 

[ERROR] GE(6029,python):2025-01-09-03:35:20.180.628 [model_manager.cc:1248]6029 LoadModelOffline: ErrorNo: 4294967295(failed) [LOAD][DEFAULT][Init][DavinciModel] failed, ret:507001. 

[ERROR] GE(6029,python):2025-01-09-03:35:20.205.142 [graph_loader.cc:141]6029 LoadModelFromData: ErrorNo: 507001() [LOAD][DEFAULT][Load][Model] failed, model_id:2. 

[ERROR] ASCENDCL(6029,python):2025-01-09-03:35:20.205.160 [model.cpp:274]6029 ModelLoadFromFileWithMem: [LOAD][DEFAULT][Model][FromData]load model from data failed, ge result[507001] 

[EVENT] IDEDH(6029,python):2025-01-09-04:49:51.622.544 [adx_server_manager.cpp:27][tid:6029]>>> start to deconstruct adx server manager 

[EVENT] DRV(6029,python):2025-01-09-04:49:51.679.466 [ascend][curpid: 6029, 6029][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=21069824. 

[EVENT] DRV(6029,python):2025-01-09-04:49:51.679.485 [ascend][curpid: 6029, 6029][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=GE, module_id=45 current_alloced_size=0, alloced_peak_size=14716928. 

[EVENT] DRV(6029,python):2025-01-09-04:49:51.679.493 [ascend][curpid: 6029, 6029][drv][devmm][_devmm_mem_stats_show 158]DEV_MEM dev0 Cached_size:0Bytes 

[EVENT] DRV(6029,python):2025-01-09-04:49:51.679.501 [ascend][curpid: 6029, 6029][drv][devmm][_devmm_mem_stats_show 152]HOST_MEM mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=11341824. 

[EVENT] DRV(6029,python):2025-01-09-04:49:51.679.508 [ascend][curpid: 6029, 6029][drv][devmm][_devmm_mem_stats_show 158]HOST_MEM Cached_size:12582912Bytes 

[EVENT] DRV(6029,python):2025-01-09-04:49:51.679.515 [ascend][curpid: 6029, 6029][drv][devmm][_devmm_mem_stats_show 158]DVPP_MEM dev0 Cached_size:0Bytes

我要发帖子