开发版上 torch-npu 跑不起来
收藏回复举报
开发版上 torch-npu 跑不起来
t('forum.solved') 已解决
发表于2025-05-21 09:57:13
0 查看

环境

硬件产品:

  • 名称:Atlas 200I DK A2
  • 芯片:310B
  • 内存:12GB
  • 算力:20TOPS

Python 环境:

(python310) root@atlas:~/workdir/py-test# python --version
Python 3.10.16

安装的 Python 包:

(python310) root@atlas:~/workdir/py-test# pip list | grep torch
torch                  2.1.0
torch-npu              2.1.0.post12
torchaudio             2.1.0
torchvision            0.16.0

CANN 环境:

(python310) root@atlas:~/workdir/py-test# ls /usr/local/Ascend/ascend-toolkit/
7.0  7.0.RC1  8.1  8.1.RC1  latest  set_env.sh
(python310) root@atlas:~/workdir/py-test# ls /usr/local/Ascend/ascend-toolkit/latest/ -l
total 32
dr-xr-xr-x 15 root root 4096 May 16 10:51 aarch64-linux
lrwxrwxrwx  1 root root   18 May 16 10:41 acllib -> ../8.1.RC1/runtime
lrwxrwxrwx  1 root root   13 May 16 10:51 arm64-linux -> aarch64-linux
lrwxrwxrwx  1 root root   14 May 16 10:43 atc -> ../8.1.RC1/atc
lrwxrwxrwx  1 root root   17 May 16 10:51 bin -> aarch64-linux/bin
lrwxrwxrwx  1 root root   19 May 16 10:43 compiler -> ../8.1.RC1/compiler
......

可以看到 latest 中的包是指向 8.0 版本的 CANN 的。

其他设置

环境变量之类的都设置好了:

(python310) root@atlas:~/workdir/py-test# echo $ASCEND_HOME_PATH/
/usr/local/Ascend/ascend-toolkit/latest/
(python310) root@atlas:~/workdir/py-test# npu-smi info
+--------------------------------------------------------------------------------------------------------+
| npu-smi 23.0.rc3                                 Version: 23.0.rc3                                     |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU     Name                  | Health          | Power(W)     Temp(C)           Hugepages-Usage(page) |
| Chip    Device                | Bus-Id          | AICore(%)    Memory-Usage(MB)                        |
+===============================+=================+======================================================+
| 0       310B1                 | OK              | 9.2          52                15    / 15            |
| 0       0                     | NA              | 0            2761 / 11577                            |
+===============================+=================+======================================================+

测试脚本

就是官方提供的脚本:

import torch
import torch_npu

x = torch.randn(2, 2).npu()
y = torch.randn(2, 2).npu()
z = x.mm(y)

print(z)

直接执行脚本得到下面的报错信息:

Traceback (most recent call last):
  File "/root/workdir/py-test/npu_test.py", line 4, in <module>
    x = torch.randn(2, 2).npu()
  File "/usr/local/miniconda3/envs/python310/lib/python3.10/site-packages/torch/utils/backend_registration.py", line 153, in wrap_tensor_to
    device_idx = _normalization_device(custom_backend_name, device)
  File "/usr/local/miniconda3/envs/python310/lib/python3.10/site-packages/torch/utils/backend_registration.py", line 109, in _normalization_device
    return _get_current_device_index()
  File "/usr/local/miniconda3/envs/python310/lib/python3.10/site-packages/torch/utils/backend_registration.py", line 103, in _get_current_device_index
    return getattr(getattr(torch, custom_backend_name), _get_device_index)()
  File "/usr/local/miniconda3/envs/python310/lib/python3.10/site-packages/torch_npu/npu/utils.py", line 87, in current_device
    torch_npu.npu._lazy_init()
  File "/usr/local/miniconda3/envs/python310/lib/python3.10/site-packages/torch_npu/npu/__init__.py", line 240, in _lazy_init
    torch_npu._C._npu_init()
RuntimeError: SetPrecisionMode:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:155 NPU function error: at_npu::native::AclSetCompileopt(aclCompileOpt::ACL_PRECISION_MODE, precision_mode), error code is 500001
[ERROR] 2025-05-21-09:45:48 (PID:12683, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The internal ACL of the system is incorrect.
        Rectify the fault based on the error information in the ascend log.
EC0010: [PID: 12683] 2025-05-21-09:45:48.098.041 Failed to import Python module [ModuleNotFoundError: No module named 'scipy'.].
        Solution: Check that all required components are properly installed and the specified Python path matches the Python installation directory. (If the path does not match the directory, run set_env.sh in the installation package.)
        TraceBack (most recent call last):
        AOE Failed to call InitCannKB[FUNC:Initialize][FILE:python_adapter_manager.cc][LINE:47]
        Failed to initialize TeConfigInfo.
        [GraphOpt][InitializeInner][InitTbeFunc] Failed to init tbe.[FUNC:InitializeTeFusion][FILE:tbe_op_store_adapter.cc][LINE:1889]
        [GraphOpt][InitializeInner][InitTeFusion]: Failed to initialize TeFusion.[FUNC:InitializeInner][FILE:tbe_op_store_adapter.cc][LINE:1856]
        [SubGraphOpt][PreCompileOp][InitAdapter] InitializeAdapter adapter [tbe_op_adapter] failed! Ret [4294967295][FUNC:InitializeAdapter][FILE:op_store_adapter_manager.cc][LINE:79]
        [SubGraphOpt][PreCompileOp][Init] Initialize op store adapter failed, OpsStoreName[tbe-custom].[FUNC:Initialize][FILE:op_store_adapter_manager.cc][LINE:120]
        [FusionMngr][Init] Op store adapter manager init failed.[FUNC:Initialize][FILE:fusion_manager.cc][LINE:115]
        PluginManager InvokeAll failed.[FUNC:Initialize][FILE:ops_kernel_manager.cc][LINE:83]
        OpsManager initialize failed.[FUNC:InnerInitialize][FILE:gelib.cc][LINE:259]
        GELib::InnerInitialize failed.[FUNC:Initialize][FILE:gelib.cc][LINE:184]
        GEInitialize failed.[FUNC:GEInitialize][FILE:ge_api.cc][LINE:371]
        [Initialize][Ge]GEInitialize failed. ge result = 4294967295[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
        [Init][Compiler]Init compiler failed[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
        [Set][Options]OpCompileProcessor init failed![FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]

大佬们看看问题在哪里?还需要排查哪些地方?

我要发帖子