模型训练打印loss报错507015
收藏回复举报
模型训练打印loss报错507015
t('forum.solved') 已解决
发表于2025-05-29 16:05:56
0 查看

问题描述:

在模型训练过程中,运行到loss.backward()之前打印loss无报错,运行后backward之后打印loss报错,报错内容如下

[W NPUStream.cpp:368] Warning: NPU warning, error code is 507015[Error]:  

[Error]: The aicore execution is abnormal.  

Rectify the fault based on the error information in the ascend log. 

EH9999: Inner Error! 

rtDeviceSynchronize execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:50] 

EH9999  wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161] 

TraceBack (most recent call last): 

 (function npuSynchronizeDevice) 

我要发帖子