华为昇腾310 c++ 调用设备初始化接口错误507033,调用接口aclrtCreateContext
说一下背景:
系统:麒麟系统
cpu:rk3588
代码在别的相同配置的机子上运行过,都很正常。就突然在这台设备上出现这个问题,重装过系统和ascend的所有软件都不行,并且发现用python调用相同接口正常,只有C++接口异常
npu-smi info:
查询结果如下
+--------------------------------------------------------------------------------------------------------+
| npu-smi 22.0.4 Version: 22.0.4 |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+===============================+=================+======================================================+
| 0 310 | OK | 12.8 50 0 / 969 |
| 0 0 | 0003:31:00.0 | 0 625 / 7759 |
+===============================+=================+======================================================+
| 1 310 | OK | 12.8 50 0 / 969 |
| 0 1 | 0002:21:00.0 | 0 625 / 7759 |
+===============================+=================+======================================================+
dmesg | grep ascend:
查询结果如下
[ 290.810853] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:3791> Release module success. (module_name="DMS"; pid=3717)
[ 290.811012] [ascend] [devdrv] [devdrv_manager_process_sign_release 778] <ACL_DEEPSORT:3717> end devdrv_get_process_sign, count = 0, docker_id = 128
[ 290.815457] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:3792> Release module success. (module_name="DEVMNG"; pid=3717)
在/root/ascend/log/debug/plog/目录下日志如下:
[ERROR] DRV(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.069 [ascend][curpid: 3717, 3717][drv][hdc][send_file_in_session 1059]Create recv_thread erro. (strerror="No such file or directory")
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.456 [process_mode_manager.cpp:300][SendAICPUPackage][tid:3717] [TsdClient][deviceId=0] drvHdcSendFile file[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend310/aicpu/Ascend310-aicpu_syskernels.tar.gz] to [/home/HwHiAiUser/hdcd/3717_Ascend310-aicpu_syskernels.tar.gz] failed, ret = 3
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.473 [process_mode_manager.cpp:209][LoadSysOpKernel][tid:3717] [TsdClient][deviceId=0] send aicpu packge to device failed
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.485 [process_mode_manager.cpp:86][Open][tid:3717] Send aicpu package to device failed.
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.496 [tsd_client.cpp:27][TsdOpen][tid:3717] TsdOpen failed, deviceId[0].
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.510 [runtime.cc:1739]3717 printf_tsd_error:report error module_type=0, module_name=E39999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.519 [runtime.cc:1739]3717 printf_tsd_error:TsdOpen failed. devId=0, tdt error=1
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.585 [runtime.cc:2268]3717 DeviceRetain:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.594 [runtime.cc:2268]3717 DeviceRetain:Start aicpu executor failed, retCode=0x7020009 devId=0
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.630 [api_impl.cc:1948]3717 NewContext:report error module_type=6, module_name=EE9999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.639 [api_impl.cc:1948]3717 NewContext:Failed to retain device, device_id=0, ts_id=0.
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.660 [api_impl.cc:2000]3717 ContextCreate:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.669 [api_impl.cc:2000]3717 ContextCreate:new context failed, devId=0, retCode=0x7010001
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.687 [logger.cc:816]3717 ContextCreate:Create context failed, devId id=0, flag=0.
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.720 [api_c.cc:2979]3717 rtCtxCreateEx:ErrCode=507033, desc=[device pointer null], InnerCode=0x7010001
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.729 [error_message_manage.cc:49]3717 FuncErrorReason:report error module_type=3, module_name=EE1001
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.743 [error_message_manage.cc:49]3717 FuncErrorReason:rtCtxCreateEx execute failed, reason=[device pointer null]
[ERROR] GE(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.763 [error_manager.cc:238]3717 ReportInterErrMessage: [Report][Error]error_code EE1001 is not internal error code
[ERROR] ASCENDCL(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.775 [context.cpp:37]3717 aclrtCreateContext: create context failed, device is 0, runtime errorCode is 507033
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
用python执行后,正常运行
dmesg | grep ascend:
查询结果如下
[ 422.168329] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.169073] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.218509] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.219036] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.220358] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.220770] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.222206] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.222627] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.223930] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.224268] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.225615] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.226042] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.227234] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.227595] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.228849] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.229213] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.233563] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.233914] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.235104] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.235534] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.236774] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.237138] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.238344] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.238674] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.240027] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.240677] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.241906] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.242382] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.243673] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.244041] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.245325] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.245695] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.247285] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.247649] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.248940] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.249315] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.250511] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.250871] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.252139] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.252667] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.253868] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.254204] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.255411] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655769ms)
[ 422.255839] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.257031] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.257414] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.258620] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.259014] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.260401] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.260772] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.261976] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.262630] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.263859] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.264203] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.265409] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655769ms)
[ 422.265756] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.266940] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.267263] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.268457] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.269062] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.270282] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.270631] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.271877] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.272552] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.273779] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.274113] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.275384] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.275735] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.277077] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.277395] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.278663] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.279234] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.280599] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.280937] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281112] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281461] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281578] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281931] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.282037] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.282390] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.283678] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.284049] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.585992] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:4029> Release module success. (module_name="DMS"; pid=4019)
[ 422.586187] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:4030> Release module success. (module_name="DEVMNG"; pid=4019)
[ 825.730879] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6383> Add release list success. (pid=6383; module_name="DMS")
[ 825.730891] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6383> Add release list success. (pid=6383; module_name="DEVMNG")
[ 825.733371] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6684> Release module success. (module_name="DMS"; pid=6383)
[ 825.733381] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6684> Release module success. (module_name="DEVMNG"; pid=6383)
[ 825.761709] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6386> Add release list success. (pid=6386; module_name="DMS")
[ 825.761714] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6386> Add release list success. (pid=6386; module_name="DEVMNG")
[ 825.761885] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6685> Release module success. (module_name="DMS"; pid=6386)
[ 825.761888] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6685> Release module success. (module_name="DEVMNG"; pid=6386)
[ 825.766587] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6382> Add release list success. (pid=6382; module_name="DMS")
[ 825.766591] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6382> Add release list success. (pid=6382; module_name="DEVMNG")
[ 825.766831] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6686> Release module success. (module_name="DMS"; pid=6382)
[ 825.766834] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6686> Release module success. (module_name="DEVMNG"; pid=6382)
[ 825.774316] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6385> Add release list success. (pid=6385; module_name="DMS")
[ 825.774321] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6385> Add release list success. (pid=6385; module_name="DEVMNG")
[ 825.774829] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6687> Release module success. (module_name="DMS"; pid=6385)
[ 825.774842] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6687> Release module success. (module_name="DEVMNG"; pid=6385)
[ 825.779757] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6380> Add release list success. (pid=6380; module_name="DMS")
[ 825.779763] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6380> Add release list success. (pid=6380; module_name="DEVMNG")
[ 825.780093] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6688> Release module success. (module_name="DMS"; pid=6380)
[ 825.780102] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6688> Release module success. (module_name="DEVMNG"; pid=6380)
[ 825.781799] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6384> Add release list success. (pid=6384; module_name="DMS")
[ 825.781803] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6384> Add release list success. (pid=6384; module_name="DEVMNG")
[ 825.782034] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6689> Release module success. (module_name="DMS"; pid=6384)
[ 825.782042] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6689> Release module success. (module_name="DEVMNG"; pid=6384)
[ 825.782361] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6381> Add release list success. (pid=6381; module_name="DMS")
[ 825.782364] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6381> Add release list success. (pid=6381; module_name="DEVMNG")
[ 825.782629] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6690> Release module success. (module_name="DMS"; pid=6381)
[ 825.782640] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6690> Release module success. (module_name="DEVMNG"; pid=6381)
[ 825.782922] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6379> Add release list success. (pid=6379; module_name="DMS")
[ 825.782925] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6379> Add release list success. (pid=6379; module_name="DEVMNG")
[ 825.783111] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6691> Release module success. (module_name="DMS"; pid=6379)
[ 825.783117] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6691> Release module success. (module_name="DEVMNG"; pid=6379)
[ 825.960531] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6343> Add release list success. (pid=6343; module_name="DMS")
[ 825.960544] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6343> Add release list success. (pid=6343; module_name="DEVMNG")
[ 825.960838] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6710> Release module success. (module_name="DMS"; pid=6343)
[ 825.960847] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6710> Release module success. (module_name="DEVMNG"; pid=6343)
[ 825.977457] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6344> Add release list success. (pid=6344; module_name="DMS")
[ 825.977471] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6344> Add release list success. (pid=6344; module_name="DEVMNG")
[ 825.977981] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6715> Release module success. (module_name="DMS"; pid=6344)
[ 825.977995] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6715> Release module success. (module_name="DEVMNG"; pid=6344)
[ 827.279859] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6717> Release module success. (module_name="DMS"; pid=6206)
[ 827.280080] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6718> Release module success. (module_name="DEVMNG"; pid=6206)
[ 852.676276] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 852.677258] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 852.677374] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 852.890163] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 857.077157] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:0:6659:6659> Get host schedule msg work time. (msg_type=3; cost_time=1431655766ms)
[ 857.077166] [ascend] [drv_pcie] [rx_msg_common_msg_process 83] <kworker/4:0:6659:6659> (dev_id=0; msg_type=5; cost_time=1431655766ms)
[ 857.157315] [ascend] [tsdrv] [tsdrv_get_dev_info 115] <python3:6760> Get dev info succeed. (virt_type=0)
[ 857.158493] [ascend] [devmm] [devmm_alloc_svm_proc_set_to_file 683] <python3:6760,6760> Devmm_set_porcess details. (status=0; proc_idx=1)
[ 857.158519] [ascend] [devmm] [devmm_svm_mmap_config_svm_proc 81] <python3:6760,6760> Devmm_map success. (hostpid=-1; devid=64; vfid=32; devpid=-1; status=0; proc_idx=1)
[ 857.186425] [ascend] [devmm] [devmm_ioctl_init_process 1731] <python3:6760,6760> Init_process details. (hostpid=6760; devid=0; vfid=0; devpid=6760; docker_id=128)
[ 857.186899] [ascend] [devmm] [devmm_add_pid_to_business_inner 101] <python3:6760,6760> Add current pid info to svm_business_info. (devid=0; pid=6760)
[ 857.187016] [ascend] [devmm] [devmm_setup_device_proc 567] <python3:6760,6760> Setup device successed. (logical_devid=0; devid=0; vfid=0; hostpid=6760; devpid=2728)
[ 857.191484] [ascend] [tsdrv] [tsdrv_sq_alloc 1188] <python3:6760> no sq available, sync from opposite side!
[ 857.195540] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 857.528524] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 858.155008] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 858.156512] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_sub_rec:6846> Release module success. (module_name="TSDRV"; pid=6760)
[ 858.255609] [ascend] [devmm] [devmm_ioctl_close_device 1825] <python3:6760,6760> Close logical_device. (logical_devid=0; phyid=0; vfid=0; hostpid=6760; devpid=2728)
[ 858.388123] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 859.248850] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 859.622528] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6854> Release module success. (module_name="DMS"; pid=6760)
[ 859.622568] [ascend] [devdrv] [devdrv_manager_process_sign_release 778] <python3:6760> end devdrv_get_process_sign, count = 0, docker_id = 128
[ 859.622874] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6855> Release module success. (module_name="DEVMNG"; pid=6760)
[ 859.665694] [ascend] [devmm] [devmm_svm_mmu_notifier_unreg 47] <python3:6760,6760> Devmm notifier unreg.(hostpid=6760; devpid=6760; devid=0; vfid=0; status=0x0; proc_idx=0x1; msg=0; other_proc=0)
[ 859.695480] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6760> Add release list success. (pid=6760; module_name="SVM")
[ 859.696028] [ascend] [devmm] [devmm_svm_release_proc 637] <davinci_recycle:6856,6856> Devmm release create work. (hostpid=6760)
[ 859.696053] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6856> Release module success. (module_name="SVM"; pid=6760)
[ 859.696072] [ascend] [devmm] [devmm_svm_release_private_proc 102] <kworker/2:0:3867,3867> Device process exited. (hostpid=6760; times=0)
[ 859.696576] [ascend] [devmm] [devmm_destory_all_heap_by_proc 1283] <kworker/2:0:3867,3867> Page statistics. (hostpid=6760; alloc_page_cnt=512; free_page_cnt=512; alloc_hugepage_cnt=0; free_huge_page_cnt=0)
[ 859.696910] [ascend] [devmm] [devmm_remove_pid_from_business 149] <kworker/2:0:3867,3867> Remove pid info from svm_business_info. (devid=0; pid=6760; business_refcount=0)
[ 859.696937] [ascend] [devmm] [devmm_free_svm_proc 289] <kworker/2:0:3867,3867> Devmm_free_process details. (hostpid=6760; devid=0; vfid=0; devpid=6760; status=0; pro_idx=1)
在/root/ascend/log/debug/plog/目录下日志如下:
[EVENT] PROFILING(6760,python3):2000-01-18-00:34:34.904.534 [msprofiler_impl.cpp:130] >>> (tid:6760) ProfNotifySetDevice called, is open: 1, devId: 0
[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:35.205.625 [model.cpp:1191]6760 aclmdlGetIndexByName: [Get][Index]get index by name failed, cannot find tensor name[ascend_mbatch_shape_data]
[EVENT] PROFILING(6760,python3):2000-01-18-00:34:35.238.116 [msprofiler_impl.cpp:130] >>> (tid:6760) ProfNotifySetDevice called, is open: 0, devId: 0
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.584 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=19718144.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.636 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=APP, module_id=33 current_alloced_size=0, alloced_peak_size=4096.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.648 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=GE, module_id=45 current_alloced_size=0, alloced_peak_size=18894848.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.659 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=ASCENDCL, module_id=48 current_alloced_size=0, alloced_peak_size=16384.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.671 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]DEV_MEM dev0 Cached_size:0Bytes
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.708 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]HOST_MEM mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=4096.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.722 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]HOST_MEM Cached_size:2097152Bytes
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.734 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DVPP_MEM dev0 mem stats(Bytes): module_name=ASCENDCL, module_id=48 current_alloced_size=0, alloced_peak_size=3338240.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.746 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]DVPP_MEM dev0 Cached_size:0Bytes
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.001 [api_impl.cc:1019]6760 DevDvppFree:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.460 [api_impl.cc:1019]6760 DevDvppFree:Invalid memoryType!, memoryType = 1.
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.692 [api_c.cc:1100]6760 rtDvppFree:ErrCode=107000, desc=[invalid value], InnerCode=0x7110001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.703 [error_message_manage.cc:47]6760 FuncErrorReason:report error module_name=EE1001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.720 [error_message_manage.cc:47]6760 FuncErrorReason:rtDvppFree execute failed, reason=[invalid value]
[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:36.962.853 [dvpp.cpp:1057]6760 acldvppFree: free device memory for dvpp failed, errorCode = 107000
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.971 [api_impl.cc:1019]6760 DevDvppFree:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.981 [api_impl.cc:1019]6760 DevDvppFree:Invalid memoryType!, memoryType = 1.
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.013 [api_c.cc:1100]6760 rtDvppFree:ErrCode=107000, desc=[invalid value], InnerCode=0x7110001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.022 [error_message_manage.cc:47]6760 FuncErrorReason:report error module_name=EE1001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.034 [error_message_manage.cc:47]6760 FuncErrorReason:rtDvppFree execute failed, reason=[invalid value]
[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:36.963.075 [dvpp.cpp:1057]6760 acldvppFree: free device memory for dvpp failed, errorCode = 107000
华为昇腾310 c++ 调用设备初始化接口错误507033,调用接口aclrtCreateContext
说一下背景:
系统:麒麟系统
cpu:rk3588
代码在别的相同配置的机子上运行过,都很正常。就突然在这台设备上出现这个问题,重装过系统和ascend的所有软件都不行,并且发现用python调用相同接口正常,只有C++接口异常
npu-smi info:
查询结果如下
+--------------------------------------------------------------------------------------------------------+
| npu-smi 22.0.4 Version: 22.0.4 |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+===============================+=================+======================================================+
| 0 310 | OK | 12.8 50 0 / 969 |
| 0 0 | 0003:31:00.0 | 0 625 / 7759 |
+===============================+=================+======================================================+
| 1 310 | OK | 12.8 50 0 / 969 |
| 0 1 | 0002:21:00.0 | 0 625 / 7759 |
+===============================+=================+======================================================+
dmesg | grep ascend:
查询结果如下
[ 290.810853] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:3791> Release module success. (module_name="DMS"; pid=3717)
[ 290.811012] [ascend] [devdrv] [devdrv_manager_process_sign_release 778] <ACL_DEEPSORT:3717> end devdrv_get_process_sign, count = 0, docker_id = 128
[ 290.815457] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:3792> Release module success. (module_name="DEVMNG"; pid=3717)
在/root/ascend/log/debug/plog/目录下日志如下:
[ERROR] DRV(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.069 [ascend][curpid: 3717, 3717][drv][hdc][send_file_in_session 1059]Create recv_thread erro. (strerror="No such file or directory")
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.456 [process_mode_manager.cpp:300][SendAICPUPackage][tid:3717] [TsdClient][deviceId=0] drvHdcSendFile file[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend310/aicpu/Ascend310-aicpu_syskernels.tar.gz] to [/home/HwHiAiUser/hdcd/3717_Ascend310-aicpu_syskernels.tar.gz] failed, ret = 3
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.473 [process_mode_manager.cpp:209][LoadSysOpKernel][tid:3717] [TsdClient][deviceId=0] send aicpu packge to device failed
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.485 [process_mode_manager.cpp:86][Open][tid:3717] Send aicpu package to device failed.
[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.496 [tsd_client.cpp:27][TsdOpen][tid:3717] TsdOpen failed, deviceId[0].
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.510 [runtime.cc:1739]3717 printf_tsd_error:report error module_type=0, module_name=E39999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.519 [runtime.cc:1739]3717 printf_tsd_error:TsdOpen failed. devId=0, tdt error=1
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.585 [runtime.cc:2268]3717 DeviceRetain:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.594 [runtime.cc:2268]3717 DeviceRetain:Start aicpu executor failed, retCode=0x7020009 devId=0
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.630 [api_impl.cc:1948]3717 NewContext:report error module_type=6, module_name=EE9999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.639 [api_impl.cc:1948]3717 NewContext:Failed to retain device, device_id=0, ts_id=0.
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.660 [api_impl.cc:2000]3717 ContextCreate:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.669 [api_impl.cc:2000]3717 ContextCreate:new context failed, devId=0, retCode=0x7010001
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.687 [logger.cc:816]3717 ContextCreate:Create context failed, devId id=0, flag=0.
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.720 [api_c.cc:2979]3717 rtCtxCreateEx:ErrCode=507033, desc=[device pointer null], InnerCode=0x7010001
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.729 [error_message_manage.cc:49]3717 FuncErrorReason:report error module_type=3, module_name=EE1001
[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.743 [error_message_manage.cc:49]3717 FuncErrorReason:rtCtxCreateEx execute failed, reason=[device pointer null]
[ERROR] GE(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.763 [error_manager.cc:238]3717 ReportInterErrMessage: [Report][Error]error_code EE1001 is not internal error code
[ERROR] ASCENDCL(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.775 [context.cpp:37]3717 aclrtCreateContext: create context failed, device is 0, runtime errorCode is 507033
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
用python执行后,正常运行
dmesg | grep ascend:
查询结果如下
[ 422.168329] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.169073] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.218509] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.219036] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.220358] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.220770] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.222206] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.222627] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.223930] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.224268] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.225615] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.226042] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.227234] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.227595] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.228849] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.229213] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.233563] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.233914] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.235104] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.235534] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.236774] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.237138] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.238344] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.238674] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.240027] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.240677] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.241906] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.242382] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.243673] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.244041] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.245325] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.245695] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.247285] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.247649] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.248940] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.249315] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.250511] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.250871] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.252139] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.252667] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.253868] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.254204] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.255411] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655769ms)
[ 422.255839] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.257031] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.257414] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.258620] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.259014] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.260401] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.260772] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.261976] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.262630] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.263859] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.264203] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.265409] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655769ms)
[ 422.265756] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.266940] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.267263] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.268457] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.269062] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.270282] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.270631] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.271877] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.272552] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.273779] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.274113] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.275384] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.275735] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.277077] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.277395] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.278663] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.279234] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.280599] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.280937] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281112] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281461] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281578] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.281931] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.282037] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.282390] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.283678] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.284049] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 422.585992] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:4029> Release module success. (module_name="DMS"; pid=4019)
[ 422.586187] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:4030> Release module success. (module_name="DEVMNG"; pid=4019)
[ 825.730879] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6383> Add release list success. (pid=6383; module_name="DMS")
[ 825.730891] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6383> Add release list success. (pid=6383; module_name="DEVMNG")
[ 825.733371] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6684> Release module success. (module_name="DMS"; pid=6383)
[ 825.733381] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6684> Release module success. (module_name="DEVMNG"; pid=6383)
[ 825.761709] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6386> Add release list success. (pid=6386; module_name="DMS")
[ 825.761714] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6386> Add release list success. (pid=6386; module_name="DEVMNG")
[ 825.761885] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6685> Release module success. (module_name="DMS"; pid=6386)
[ 825.761888] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6685> Release module success. (module_name="DEVMNG"; pid=6386)
[ 825.766587] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6382> Add release list success. (pid=6382; module_name="DMS")
[ 825.766591] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6382> Add release list success. (pid=6382; module_name="DEVMNG")
[ 825.766831] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6686> Release module success. (module_name="DMS"; pid=6382)
[ 825.766834] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6686> Release module success. (module_name="DEVMNG"; pid=6382)
[ 825.774316] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6385> Add release list success. (pid=6385; module_name="DMS")
[ 825.774321] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6385> Add release list success. (pid=6385; module_name="DEVMNG")
[ 825.774829] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6687> Release module success. (module_name="DMS"; pid=6385)
[ 825.774842] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6687> Release module success. (module_name="DEVMNG"; pid=6385)
[ 825.779757] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6380> Add release list success. (pid=6380; module_name="DMS")
[ 825.779763] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6380> Add release list success. (pid=6380; module_name="DEVMNG")
[ 825.780093] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6688> Release module success. (module_name="DMS"; pid=6380)
[ 825.780102] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6688> Release module success. (module_name="DEVMNG"; pid=6380)
[ 825.781799] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6384> Add release list success. (pid=6384; module_name="DMS")
[ 825.781803] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6384> Add release list success. (pid=6384; module_name="DEVMNG")
[ 825.782034] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6689> Release module success. (module_name="DMS"; pid=6384)
[ 825.782042] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6689> Release module success. (module_name="DEVMNG"; pid=6384)
[ 825.782361] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6381> Add release list success. (pid=6381; module_name="DMS")
[ 825.782364] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6381> Add release list success. (pid=6381; module_name="DEVMNG")
[ 825.782629] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6690> Release module success. (module_name="DMS"; pid=6381)
[ 825.782640] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6690> Release module success. (module_name="DEVMNG"; pid=6381)
[ 825.782922] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6379> Add release list success. (pid=6379; module_name="DMS")
[ 825.782925] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6379> Add release list success. (pid=6379; module_name="DEVMNG")
[ 825.783111] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6691> Release module success. (module_name="DMS"; pid=6379)
[ 825.783117] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6691> Release module success. (module_name="DEVMNG"; pid=6379)
[ 825.960531] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6343> Add release list success. (pid=6343; module_name="DMS")
[ 825.960544] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6343> Add release list success. (pid=6343; module_name="DEVMNG")
[ 825.960838] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6710> Release module success. (module_name="DMS"; pid=6343)
[ 825.960847] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6710> Release module success. (module_name="DEVMNG"; pid=6343)
[ 825.977457] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6344> Add release list success. (pid=6344; module_name="DMS")
[ 825.977471] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6344> Add release list success. (pid=6344; module_name="DEVMNG")
[ 825.977981] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6715> Release module success. (module_name="DMS"; pid=6344)
[ 825.977995] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6715> Release module success. (module_name="DEVMNG"; pid=6344)
[ 827.279859] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6717> Release module success. (module_name="DMS"; pid=6206)
[ 827.280080] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6718> Release module success. (module_name="DEVMNG"; pid=6206)
[ 852.676276] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 852.677258] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 852.677374] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 852.890163] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 857.077157] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:0:6659:6659> Get host schedule msg work time. (msg_type=3; cost_time=1431655766ms)
[ 857.077166] [ascend] [drv_pcie] [rx_msg_common_msg_process 83] <kworker/4:0:6659:6659> (dev_id=0; msg_type=5; cost_time=1431655766ms)
[ 857.157315] [ascend] [tsdrv] [tsdrv_get_dev_info 115] <python3:6760> Get dev info succeed. (virt_type=0)
[ 857.158493] [ascend] [devmm] [devmm_alloc_svm_proc_set_to_file 683] <python3:6760,6760> Devmm_set_porcess details. (status=0; proc_idx=1)
[ 857.158519] [ascend] [devmm] [devmm_svm_mmap_config_svm_proc 81] <python3:6760,6760> Devmm_map success. (hostpid=-1; devid=64; vfid=32; devpid=-1; status=0; proc_idx=1)
[ 857.186425] [ascend] [devmm] [devmm_ioctl_init_process 1731] <python3:6760,6760> Init_process details. (hostpid=6760; devid=0; vfid=0; devpid=6760; docker_id=128)
[ 857.186899] [ascend] [devmm] [devmm_add_pid_to_business_inner 101] <python3:6760,6760> Add current pid info to svm_business_info. (devid=0; pid=6760)
[ 857.187016] [ascend] [devmm] [devmm_setup_device_proc 567] <python3:6760,6760> Setup device successed. (logical_devid=0; devid=0; vfid=0; hostpid=6760; devpid=2728)
[ 857.191484] [ascend] [tsdrv] [tsdrv_sq_alloc 1188] <python3:6760> no sq available, sync from opposite side!
[ 857.195540] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 857.528524] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 858.155008] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 858.156512] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_sub_rec:6846> Release module success. (module_name="TSDRV"; pid=6760)
[ 858.255609] [ascend] [devmm] [devmm_ioctl_close_device 1825] <python3:6760,6760> Close logical_device. (logical_devid=0; phyid=0; vfid=0; hostpid=6760; devpid=2728)
[ 858.388123] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 859.248850] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms)
[ 859.622528] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6854> Release module success. (module_name="DMS"; pid=6760)
[ 859.622568] [ascend] [devdrv] [devdrv_manager_process_sign_release 778] <python3:6760> end devdrv_get_process_sign, count = 0, docker_id = 128
[ 859.622874] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6855> Release module success. (module_name="DEVMNG"; pid=6760)
[ 859.665694] [ascend] [devmm] [devmm_svm_mmu_notifier_unreg 47] <python3:6760,6760> Devmm notifier unreg.(hostpid=6760; devpid=6760; devid=0; vfid=0; status=0x0; proc_idx=0x1; msg=0; other_proc=0)
[ 859.695480] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6760> Add release list success. (pid=6760; module_name="SVM")
[ 859.696028] [ascend] [devmm] [devmm_svm_release_proc 637] <davinci_recycle:6856,6856> Devmm release create work. (hostpid=6760)
[ 859.696053] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6856> Release module success. (module_name="SVM"; pid=6760)
[ 859.696072] [ascend] [devmm] [devmm_svm_release_private_proc 102] <kworker/2:0:3867,3867> Device process exited. (hostpid=6760; times=0)
[ 859.696576] [ascend] [devmm] [devmm_destory_all_heap_by_proc 1283] <kworker/2:0:3867,3867> Page statistics. (hostpid=6760; alloc_page_cnt=512; free_page_cnt=512; alloc_hugepage_cnt=0; free_huge_page_cnt=0)
[ 859.696910] [ascend] [devmm] [devmm_remove_pid_from_business 149] <kworker/2:0:3867,3867> Remove pid info from svm_business_info. (devid=0; pid=6760; business_refcount=0)
[ 859.696937] [ascend] [devmm] [devmm_free_svm_proc 289] <kworker/2:0:3867,3867> Devmm_free_process details. (hostpid=6760; devid=0; vfid=0; devpid=6760; status=0; pro_idx=1)
在/root/ascend/log/debug/plog/目录下日志如下:
[EVENT] PROFILING(6760,python3):2000-01-18-00:34:34.904.534 [msprofiler_impl.cpp:130] >>> (tid:6760) ProfNotifySetDevice called, is open: 1, devId: 0
[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:35.205.625 [model.cpp:1191]6760 aclmdlGetIndexByName: [Get][Index]get index by name failed, cannot find tensor name[ascend_mbatch_shape_data]
[EVENT] PROFILING(6760,python3):2000-01-18-00:34:35.238.116 [msprofiler_impl.cpp:130] >>> (tid:6760) ProfNotifySetDevice called, is open: 0, devId: 0
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.584 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=19718144.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.636 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=APP, module_id=33 current_alloced_size=0, alloced_peak_size=4096.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.648 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=GE, module_id=45 current_alloced_size=0, alloced_peak_size=18894848.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.659 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=ASCENDCL, module_id=48 current_alloced_size=0, alloced_peak_size=16384.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.671 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]DEV_MEM dev0 Cached_size:0Bytes
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.708 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]HOST_MEM mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=4096.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.722 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]HOST_MEM Cached_size:2097152Bytes
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.734 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DVPP_MEM dev0 mem stats(Bytes): module_name=ASCENDCL, module_id=48 current_alloced_size=0, alloced_peak_size=3338240.
[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.746 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]DVPP_MEM dev0 Cached_size:0Bytes
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.001 [api_impl.cc:1019]6760 DevDvppFree:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.460 [api_impl.cc:1019]6760 DevDvppFree:Invalid memoryType!, memoryType = 1.
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.692 [api_c.cc:1100]6760 rtDvppFree:ErrCode=107000, desc=[invalid value], InnerCode=0x7110001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.703 [error_message_manage.cc:47]6760 FuncErrorReason:report error module_name=EE1001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.720 [error_message_manage.cc:47]6760 FuncErrorReason:rtDvppFree execute failed, reason=[invalid value]
[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:36.962.853 [dvpp.cpp:1057]6760 acldvppFree: free device memory for dvpp failed, errorCode = 107000
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.971 [api_impl.cc:1019]6760 DevDvppFree:report error module_type=0, module_name=EE9999
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.981 [api_impl.cc:1019]6760 DevDvppFree:Invalid memoryType!, memoryType = 1.
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.013 [api_c.cc:1100]6760 rtDvppFree:ErrCode=107000, desc=[invalid value], InnerCode=0x7110001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.022 [error_message_manage.cc:47]6760 FuncErrorReason:report error module_name=EE1001
[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.034 [error_message_manage.cc:47]6760 FuncErrorReason:rtDvppFree execute failed, reason=[invalid value]
[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:36.963.075 [dvpp.cpp:1057]6760 acldvppFree: free device memory for dvpp failed, errorCode = 107000