一、问题现象（附报错日志上下文）：

(sp) [ma-user ModelLink]$python tools/checkpoint/convert_ckpt.py \ 

>     --model-type GPT \ 

>     --loader llama2_hf \ 

>     --saver megatron \ 

>     --target-tensor-parallel-size 1 \ 

>     --target-pipeline-parallel-size 2 \ 

>     --params-dtype bf16 \ 

>     --add-qkv-bias \ 

>     --load-dir ./model_from_hf/qwen15-4b-hf/ \ 

>     --save-dir ./model_weights/qwen15-4b-hf-v0.1-tp1-pp2/ \ 

>     --tokenizer-model ./model_from_hf/qwen15-4b-hf/tokenizer.json 

/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/torch_npu/dynamo/__init__.py:18: UserWarning: Register eager implementation for the 'npu' backend of dynamo, as torch_npu was not compiled with torchair. 

  warnings.warn( 

/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/torch_npu/contrib/transfer_to_npu.py:208: ImportWarning: 

    ************************************************************************************************************* 

    The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now.. 

    The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now.. 

    The backend in torch.distributed.init_process_group set to hccl now.. 

    The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now.. 

    The device parameters have been replaced with npu in the function below: 

    torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.autocast, torch.load, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.nn.Module.to, torch.nn.Module.to_empty 

    ************************************************************************************************************* 

  warnings.warn(msg, ImportWarning) 

/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/torch/utils/cpp_extension.py:28: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 

  from pkg_resources import packaging  # type: ignore[attr-defined] 

Zarr-based strategies will not be registered because of missing packages 

Traceback (most recent call last): 

  File "tools/checkpoint/convert_ckpt.py", line 23, in <module> 

    import modellink 

  File "/home/ma-user/work/sp/fbig/ModelLink/modellink/__init__.py", line 25, in <module> 

    from .model_adaptor import exe_adaptor 

  File "/home/ma-user/work/sp/fbig/ModelLink/modellink/model_adaptor.py", line 16, in <module> 

    import megatron 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/__init__.py", line 15, in <module> 

    from .initialize  import initialize_megatron 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/initialize.py", line 18, in <module> 

    from megatron.arguments import parse_args, validate_args 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/arguments.py", line 16, in <module> 

    from megatron.core.models.retro import RetroConfig 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/retro/__init__.py", line 4, in <module> 

    from .decoder_spec import get_retro_decoder_block_spec 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/retro/decoder_spec.py", line 5, in <module> 

    from megatron.core.models.gpt.gpt_layer_specs import ( 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/gpt/__init__.py", line 1, in <module> 

    from .gpt_model import GPTModel 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/gpt/gpt_model.py", line 15, in <module> 

    from megatron.core.transformer.transformer_block import TransformerBlock 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/transformer/transformer_block.py", line 13, in <module> 

    from megatron.core.transformer.custom_layers.transformer_engine import TENorm 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/transformer/custom_layers/transformer_engine.py", line 6, in <module> 

    import transformer_engine as te 

ModuleNotFoundError: No module named 'transformer_engine' 

二、软件版本:
-- CANN 版本 (e.g., CANN 3.0.x，5.x.x): 8.0.RC1
--Tensorflow/Pytorch/MindSpore 版本:torch=2.1.0
--Python 版本 (e.g., Python 3.7.5):3.8.20
-- MindStudio版本 (e.g., MindStudio 2.0.0 (beta3)):\
--操作系统版本 (e.g., Ubuntu 18.04):EulerOS 2.0 (SP8)

三、测试步骤：

(1)pip安装:pip install transformer-engine 权重转换仍无法运行，并出现新的报错：OSError: libcudnn.so: cannot open shared object file: No such file or directory 

/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/torch_npu/dynamo/__init__.py:18: UserWarning: Register eager implementation for the 'npu' backend of dynamo, as torch_npu was not compiled with torchair. 

  warnings.warn( 

/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/torch_npu/contrib/transfer_to_npu.py:208: ImportWarning: 

    ************************************************************************************************************* 

    The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now.. 

    The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now.. 

    The backend in torch.distributed.init_process_group set to hccl now.. 

    The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now.. 

    The device parameters have been replaced with npu in the function below: 

    torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.autocast, torch.load, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.nn.Module.to, torch.nn.Module.to_empty 

    ************************************************************************************************************* 

 

  warnings.warn(msg, ImportWarning) 

/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/torch/utils/cpp_extension.py:28: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 

  from pkg_resources import packaging  # type: ignore[attr-defined] 

Zarr-based strategies will not be registered because of missing packages 

Traceback (most recent call last): 

  File "tools/checkpoint/convert_ckpt.py", line 23, in <module> 

    import modellink 

  File "/home/ma-user/work/sp/fbig/ModelLink/modellink/__init__.py", line 25, in <module> 

    from .model_adaptor import exe_adaptor 

  File "/home/ma-user/work/sp/fbig/ModelLink/modellink/model_adaptor.py", line 16, in <module> 

    import megatron 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/__init__.py", line 15, in <module> 

    from .initialize  import initialize_megatron 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/initialize.py", line 18, in <module> 

    from megatron.arguments import parse_args, validate_args 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/arguments.py", line 16, in <module> 

    from megatron.core.models.retro import RetroConfig 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/retro/__init__.py", line 4, in <module> 

    from .decoder_spec import get_retro_decoder_block_spec 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/retro/decoder_spec.py", line 5, in <module> 

    from megatron.core.models.gpt.gpt_layer_specs import ( 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/gpt/__init__.py", line 1, in <module> 

    from .gpt_model import GPTModel 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/models/gpt/gpt_model.py", line 15, in <module> 

    from megatron.core.transformer.transformer_block import TransformerBlock 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/transformer/transformer_block.py", line 13, in <module> 

    from megatron.core.transformer.custom_layers.transformer_engine import TENorm 

  File "/home/ma-user/work/sp/fbig/ModelLink/megatron/core/transformer/custom_layers/transformer_engine.py", line 6, in <module> 

    import transformer_engine as te 

  File "/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/transformer_engine/__init__.py", line 10, in <module> 

    import transformer_engine.common 

  File "/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/transformer_engine/common/__init__.py", line 120, in <module> 

    _CUDNN_LIB_CTYPES = _load_cudnn() 

  File "/home/ma-user/anaconda3/envs/sp/lib/python3.8/site-packages/transformer_engine/common/__init__.py", line 79, in _load_cudnn 

    return ctypes.CDLL(f"libcudnn.{_get_sys_extension()}", mode=ctypes.RTLD_GLOBAL) 

  File "/home/ma-user/anaconda3/envs/sp/lib/python3.8/ctypes/__init__.py", line 373, in __init__ 

    self._handle = _dlopen(self._name, mode) 

OSError: libcudnn.so: cannot open shared object file: No such file or directory 

请问产生transformer_engine包报错的原因是什么以及有效的解决方法？