在使用晟腾pytorch中碰到问题
收藏回复举报
在使用晟腾pytorch中碰到问题
t('forum.solved') 已解决
发表于2024-11-27 15:43:30
0 查看

在使用晟腾的pytorch在910B上做resnet50的训练中。在启动optimizer.step()过程中报如下错误:

File "/usr/local/lib/python3.10/site-packages/torch/optim/sgd.py", line 293, in _multi_tensor_sgd 

    device_grads = torch._foreach_add(device_grads, device_params, alpha=weight_decay) 

RuntimeError: call aclnnForeachAddList failed, detail:EZ1001: [PID: 1165457] 2024-11-27-15:18:33.749.618 Io input dtype or format not support, get io input info is x1(DT_FLOAT, ND) x2(DT_FLOAT, ND) alpha(UNDEFINED, ND) out(DT_FLOAT, ND) but support list is: 

        TraceBack (most recent call last): 

        DataTypeToSerialString: datatype not support 381622840[FUNC:DataTypeToSerialString][FILE:type_utils.cc][LINE:288] 

        Io input dtype or format not support, get io input info is x1(DT_FLOAT, ND) x2(DT_FLOAT, ND) alpha(UNDEFINED, ND) out(DT_FLOAT, ND) but support list is: 

        SupportInfo[0] x1(DT_FLOAT16, ND) x2(DT_FLOAT16, ND) alpha(DT_FLOAT, ND) out(DT_FLOAT16, ND)  

        SupportInfo[1] x1(DT_FLOAT, ND) x2(DT_FLOAT, ND) alpha(DT_FLOAT, ND) out(DT_FLOAT, ND)  

        SupportInfo[2] x1(DT_INT32, ND) x2(DT_INT32, ND) alpha(DT_INT64, ND) out(DT_INT32, ND)  

        SupportInfo[3] x1(DT_BFLOAT16, ND) x2(DT_BFLOAT16, ND) alpha(DT_FLOAT, ND) out(DT_BFLOAT16, ND)  

        Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed

本帖最后由 匿名用户2024/11/27 16:30:01 编辑

我要发帖子