华为昇腾310 c++ 调用设备初始化接口错误507033,调用接口aclrtCreateContext 
收藏回复举报
华为昇腾310 c++ 调用设备初始化接口错误507033,调用接口aclrtCreateContext 
t('forum.solved') 已解决
新人帖
发表于2023-08-28 16:55:33
0 查看

华为昇腾310 c++ 调用设备初始化接口错误507033,调用接口aclrtCreateContext 

说一下背景: 

系统:麒麟系统 

cpu:rk3588 

代码在别的相同配置的机子上运行过,都很正常。就突然在这台设备上出现这个问题,重装过系统和ascend的所有软件都不行,并且发现用python调用相同接口正常,只有C++接口异常 

 

npu-smi info: 

查询结果如下 

+--------------------------------------------------------------------------------------------------------+ 

| npu-smi 22.0.4                                   Version: 22.0.4                                       | 

+-------------------------------+-----------------+------------------------------------------------------+ 

| NPU     Name                  | Health          | Power(W)     Temp(C)           Hugepages-Usage(page) | 

| Chip    Device                | Bus-Id          | AICore(%)    Memory-Usage(MB)                        | 

+===============================+=================+======================================================+ 

| 0       310                   | OK              | 12.8         50                0    / 969            | 

| 0       0                     | 0003:31:00.0    | 0            625  / 7759                             | 

+===============================+=================+======================================================+ 

| 1       310                   | OK              | 12.8         50                0    / 969            | 

| 0       1                     | 0002:21:00.0    | 0            625  / 7759                             | 

+===============================+=================+======================================================+ 

 

dmesg | grep ascend: 

查询结果如下 

[  290.810853] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:3791> Release module success. (module_name="DMS"; pid=3717) 

[  290.811012] [ascend] [devdrv] [devdrv_manager_process_sign_release 778] <ACL_DEEPSORT:3717> end devdrv_get_process_sign, count = 0, docker_id = 128 

[  290.815457] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:3792> Release module success. (module_name="DEVMNG"; pid=3717) 

 

在/root/ascend/log/debug/plog/目录下日志如下: 

[ERROR] DRV(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.069 [ascend][curpid: 3717, 3717][drv][hdc][send_file_in_session 1059]Create recv_thread erro. (strerror="No such file or directory") 

[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.456 [process_mode_manager.cpp:300][SendAICPUPackage][tid:3717] [TsdClient][deviceId=0] drvHdcSendFile file[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend310/aicpu/Ascend310-aicpu_syskernels.tar.gz] to [/home/HwHiAiUser/hdcd/3717_Ascend310-aicpu_syskernels.tar.gz] failed, ret = 3 

[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.473 [process_mode_manager.cpp:209][LoadSysOpKernel][tid:3717] [TsdClient][deviceId=0] send aicpu packge to device failed 

[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.485 [process_mode_manager.cpp:86][Open][tid:3717] Send aicpu package to device failed. 

[ERROR] TDT(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.496 [tsd_client.cpp:27][TsdOpen][tid:3717] TsdOpen failed, deviceId[0]. 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.510 [runtime.cc:1739]3717 printf_tsd_error:report error module_type=0, module_name=E39999 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.519 [runtime.cc:1739]3717 printf_tsd_error:TsdOpen failed. devId=0, tdt error=1 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.585 [runtime.cc:2268]3717 DeviceRetain:report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.594 [runtime.cc:2268]3717 DeviceRetain:Start aicpu executor failed, retCode=0x7020009 devId=0 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.630 [api_impl.cc:1948]3717 NewContext:report error module_type=6, module_name=EE9999 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.639 [api_impl.cc:1948]3717 NewContext:Failed to retain device, device_id=0, ts_id=0. 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.660 [api_impl.cc:2000]3717 ContextCreate:report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.669 [api_impl.cc:2000]3717 ContextCreate:new context failed, devId=0, retCode=0x7010001 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.687 [logger.cc:816]3717 ContextCreate:Create context failed, devId id=0, flag=0. 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.720 [api_c.cc:2979]3717 rtCtxCreateEx:ErrCode=507033, desc=[device pointer null], InnerCode=0x7010001 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.729 [error_message_manage.cc:49]3717 FuncErrorReason:report error module_type=3, module_name=EE1001 

[ERROR] RUNTIME(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.743 [error_message_manage.cc:49]3717 FuncErrorReason:rtCtxCreateEx execute failed, reason=[device pointer null] 

[ERROR] GE(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.763 [error_manager.cc:238]3717 ReportInterErrMessage: [Report][Error]error_code EE1001 is not internal error code 

[ERROR] ASCENDCL(3717,ACL_DEEPSORT):2000-01-18-00:25:07.319.775 [context.cpp:37]3717 aclrtCreateContext: create context failed, device is 0, runtime errorCode is 507033 

 

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 

 

用python执行后,正常运行 

dmesg | grep ascend: 

查询结果如下 

[  422.168329] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.169073] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.218509] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.219036] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.220358] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.220770] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.222206] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.222627] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.223930] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.224268] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.225615] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.226042] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.227234] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.227595] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.228849] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.229213] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.233563] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.233914] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.235104] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.235534] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.236774] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.237138] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.238344] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.238674] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.240027] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.240677] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.241906] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.242382] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.243673] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.244041] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.245325] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.245695] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.247285] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.247649] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.248940] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.249315] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.250511] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.250871] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.252139] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.252667] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.253868] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.254204] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.255411] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655769ms) 

[  422.255839] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.257031] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.257414] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.258620] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.259014] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.260401] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.260772] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.261976] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.262630] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.263859] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.264203] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.265409] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655769ms) 

[  422.265756] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.266940] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.267263] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.268457] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.269062] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.270282] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.270631] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.271877] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.272552] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.273779] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.274113] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.275384] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.275735] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.277077] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.277395] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.278663] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.279234] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:2:179:179> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.280599] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.280937] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.281112] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.281461] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.281578] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.281931] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.282037] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.282390] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.283678] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.284049] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  422.585992] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:4029> Release module success. (module_name="DMS"; pid=4019) 

[  422.586187] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:4030> Release module success. (module_name="DEVMNG"; pid=4019) 

[  825.730879] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6383> Add release list success. (pid=6383; module_name="DMS") 

[  825.730891] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6383> Add release list success. (pid=6383; module_name="DEVMNG") 

[  825.733371] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6684> Release module success. (module_name="DMS"; pid=6383) 

[  825.733381] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6684> Release module success. (module_name="DEVMNG"; pid=6383) 

[  825.761709] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6386> Add release list success. (pid=6386; module_name="DMS") 

[  825.761714] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6386> Add release list success. (pid=6386; module_name="DEVMNG") 

[  825.761885] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6685> Release module success. (module_name="DMS"; pid=6386) 

[  825.761888] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6685> Release module success. (module_name="DEVMNG"; pid=6386) 

[  825.766587] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6382> Add release list success. (pid=6382; module_name="DMS") 

[  825.766591] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6382> Add release list success. (pid=6382; module_name="DEVMNG") 

[  825.766831] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6686> Release module success. (module_name="DMS"; pid=6382) 

[  825.766834] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6686> Release module success. (module_name="DEVMNG"; pid=6382) 

[  825.774316] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6385> Add release list success. (pid=6385; module_name="DMS") 

[  825.774321] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6385> Add release list success. (pid=6385; module_name="DEVMNG") 

[  825.774829] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6687> Release module success. (module_name="DMS"; pid=6385) 

[  825.774842] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6687> Release module success. (module_name="DEVMNG"; pid=6385) 

[  825.779757] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6380> Add release list success. (pid=6380; module_name="DMS") 

[  825.779763] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6380> Add release list success. (pid=6380; module_name="DEVMNG") 

[  825.780093] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6688> Release module success. (module_name="DMS"; pid=6380) 

[  825.780102] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6688> Release module success. (module_name="DEVMNG"; pid=6380) 

[  825.781799] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6384> Add release list success. (pid=6384; module_name="DMS") 

[  825.781803] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6384> Add release list success. (pid=6384; module_name="DEVMNG") 

[  825.782034] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6689> Release module success. (module_name="DMS"; pid=6384) 

[  825.782042] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6689> Release module success. (module_name="DEVMNG"; pid=6384) 

[  825.782361] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6381> Add release list success. (pid=6381; module_name="DMS") 

[  825.782364] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6381> Add release list success. (pid=6381; module_name="DEVMNG") 

[  825.782629] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6690> Release module success. (module_name="DMS"; pid=6381) 

[  825.782640] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6690> Release module success. (module_name="DEVMNG"; pid=6381) 

[  825.782922] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6379> Add release list success. (pid=6379; module_name="DMS") 

[  825.782925] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6379> Add release list success. (pid=6379; module_name="DEVMNG") 

[  825.783111] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6691> Release module success. (module_name="DMS"; pid=6379) 

[  825.783117] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6691> Release module success. (module_name="DEVMNG"; pid=6379) 

[  825.960531] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6343> Add release list success. (pid=6343; module_name="DMS") 

[  825.960544] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <alogFlush:6343> Add release list success. (pid=6343; module_name="DEVMNG") 

[  825.960838] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6710> Release module success. (module_name="DMS"; pid=6343) 

[  825.960847] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6710> Release module success. (module_name="DEVMNG"; pid=6343) 

[  825.977457] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6344> Add release list success. (pid=6344; module_name="DMS") 

[  825.977471] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6344> Add release list success. (pid=6344; module_name="DEVMNG") 

[  825.977981] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6715> Release module success. (module_name="DMS"; pid=6344) 

[  825.977995] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6715> Release module success. (module_name="DEVMNG"; pid=6344) 

[  827.279859] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6717> Release module success. (module_name="DMS"; pid=6206) 

[  827.280080] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6718> Release module success. (module_name="DEVMNG"; pid=6206) 

[  852.676276] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  852.677258] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  852.677374] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  852.890163] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  857.077157] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/4:0:6659:6659> Get host schedule msg work time. (msg_type=3; cost_time=1431655766ms) 

[  857.077166] [ascend] [drv_pcie] [rx_msg_common_msg_process 83] <kworker/4:0:6659:6659> (dev_id=0; msg_type=5; cost_time=1431655766ms) 

[  857.157315] [ascend] [tsdrv] [tsdrv_get_dev_info 115] <python3:6760> Get dev info succeed. (virt_type=0) 

[  857.158493] [ascend] [devmm] [devmm_alloc_svm_proc_set_to_file 683] <python3:6760,6760> Devmm_set_porcess details. (status=0; proc_idx=1) 

[  857.158519] [ascend] [devmm] [devmm_svm_mmap_config_svm_proc 81] <python3:6760,6760> Devmm_map success. (hostpid=-1; devid=64; vfid=32; devpid=-1; status=0; proc_idx=1) 

[  857.186425] [ascend] [devmm] [devmm_ioctl_init_process 1731] <python3:6760,6760> Init_process details. (hostpid=6760; devid=0; vfid=0; devpid=6760; docker_id=128) 

[  857.186899] [ascend] [devmm] [devmm_add_pid_to_business_inner 101] <python3:6760,6760> Add current pid info to svm_business_info. (devid=0; pid=6760) 

[  857.187016] [ascend] [devmm] [devmm_setup_device_proc 567] <python3:6760,6760> Setup device successed. (logical_devid=0; devid=0; vfid=0; hostpid=6760; devpid=2728) 

[  857.191484] [ascend] [tsdrv] [tsdrv_sq_alloc 1188] <python3:6760> no sq available, sync from opposite side! 

[  857.195540] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  857.528524] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  858.155008] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  858.156512] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_sub_rec:6846> Release module success. (module_name="TSDRV"; pid=6760) 

[  858.255609] [ascend] [devmm] [devmm_ioctl_close_device 1825] <python3:6760,6760> Close logical_device. (logical_devid=0; phyid=0; vfid=0; hostpid=6760; devpid=2728) 

[  858.388123] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  859.248850] [ascend] [drv_pcie] [devdrv_non_trans_rx_msg_task_resq_record 659] <kworker/0:2:447:447> Get host schedule msg work time. (msg_type=6; cost_time=1431655766ms) 

[  859.622528] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6854> Release module success. (module_name="DMS"; pid=6760) 

[  859.622568] [ascend] [devdrv] [devdrv_manager_process_sign_release 778] <python3:6760> end devdrv_get_process_sign, count = 0, docker_id = 128 

[  859.622874] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6855> Release module success. (module_name="DEVMNG"; pid=6760) 

[  859.665694] [ascend] [devmm] [devmm_svm_mmu_notifier_unreg 47] <python3:6760,6760> Devmm notifier unreg.(hostpid=6760; devpid=6760; devid=0; vfid=0; status=0x0; proc_idx=0x1; msg=0; other_proc=0) 

[  859.695480] [ascend] [ascend_dev] [drv_ascend_add_release_list_all 774] <python3:6760> Add release list success. (pid=6760; module_name="SVM") 

[  859.696028] [ascend] [devmm] [devmm_svm_release_proc 637] <davinci_recycle:6856,6856> Devmm release create work. (hostpid=6760) 

[  859.696053] [ascend] [ascend_dev] [drv_ascend_intf_sub_call_release 374] <davinci_recycle:6856> Release module success. (module_name="SVM"; pid=6760) 

[  859.696072] [ascend] [devmm] [devmm_svm_release_private_proc 102] <kworker/2:0:3867,3867> Device process exited. (hostpid=6760; times=0) 

[  859.696576] [ascend] [devmm] [devmm_destory_all_heap_by_proc 1283] <kworker/2:0:3867,3867> Page statistics. (hostpid=6760; alloc_page_cnt=512; free_page_cnt=512; alloc_hugepage_cnt=0; free_huge_page_cnt=0) 

[  859.696910] [ascend] [devmm] [devmm_remove_pid_from_business 149] <kworker/2:0:3867,3867> Remove pid info from svm_business_info. (devid=0; pid=6760; business_refcount=0) 

[  859.696937] [ascend] [devmm] [devmm_free_svm_proc 289] <kworker/2:0:3867,3867> Devmm_free_process details. (hostpid=6760; devid=0; vfid=0; devpid=6760; status=0; pro_idx=1) 

 

在/root/ascend/log/debug/plog/目录下日志如下: 

[EVENT] PROFILING(6760,python3):2000-01-18-00:34:34.904.534 [msprofiler_impl.cpp:130] >>> (tid:6760) ProfNotifySetDevice called, is open: 1, devId: 0 

[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:35.205.625 [model.cpp:1191]6760 aclmdlGetIndexByName: [Get][Index]get index by name failed, cannot find tensor name[ascend_mbatch_shape_data] 

[EVENT] PROFILING(6760,python3):2000-01-18-00:34:35.238.116 [msprofiler_impl.cpp:130] >>> (tid:6760) ProfNotifySetDevice called, is open: 0, devId: 0 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.584 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=19718144. 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.636 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=APP, module_id=33 current_alloced_size=0, alloced_peak_size=4096. 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.648 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=GE, module_id=45 current_alloced_size=0, alloced_peak_size=18894848. 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.659 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DEV_MEM dev0 mem stats(Bytes): module_name=ASCENDCL, module_id=48 current_alloced_size=0, alloced_peak_size=16384. 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.671 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]DEV_MEM dev0 Cached_size:0Bytes 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.708 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]HOST_MEM mem stats(Bytes): module_name=RUNTIME, module_id=7 current_alloced_size=0, alloced_peak_size=4096. 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.722 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]HOST_MEM Cached_size:2097152Bytes 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.734 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 152]DVPP_MEM dev0 mem stats(Bytes): module_name=ASCENDCL, module_id=48 current_alloced_size=0, alloced_peak_size=3338240. 

[EVENT] DRV(6760,python3):2000-01-18-00:34:36.095.746 [ascend][curpid: 6760, 6760][drv][devmm][_devmm_mem_stats_show 158]DVPP_MEM dev0 Cached_size:0Bytes 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.001 [api_impl.cc:1019]6760 DevDvppFree:report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.460 [api_impl.cc:1019]6760 DevDvppFree:Invalid memoryType!, memoryType = 1. 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.692 [api_c.cc:1100]6760 rtDvppFree:ErrCode=107000, desc=[invalid value], InnerCode=0x7110001 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.703 [error_message_manage.cc:47]6760 FuncErrorReason:report error module_name=EE1001 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.720 [error_message_manage.cc:47]6760 FuncErrorReason:rtDvppFree execute failed, reason=[invalid value] 

[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:36.962.853 [dvpp.cpp:1057]6760 acldvppFree: free device memory for dvpp failed, errorCode = 107000 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.971 [api_impl.cc:1019]6760 DevDvppFree:report error module_type=0, module_name=EE9999 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.962.981 [api_impl.cc:1019]6760 DevDvppFree:Invalid memoryType!, memoryType = 1. 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.013 [api_c.cc:1100]6760 rtDvppFree:ErrCode=107000, desc=[invalid value], InnerCode=0x7110001 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.022 [error_message_manage.cc:47]6760 FuncErrorReason:report error module_name=EE1001 

[ERROR] RUNTIME(6760,python3):2000-01-18-00:34:36.963.034 [error_message_manage.cc:47]6760 FuncErrorReason:rtDvppFree execute failed, reason=[invalid value] 

[ERROR] ASCENDCL(6760,python3):2000-01-18-00:34:36.963.075 [dvpp.cpp:1057]6760 acldvppFree: free device memory for dvpp failed, errorCode = 107000 

 

我要发帖子