为更快的帮您定位问题，推荐您用以下模板反馈：
1、出现问题时，您做了哪些操作？
答复：
Ascend Docker Runtime最小化验证成功：
参考mis-tei操作的docker指令：docker run -u root -e ASCEND_VISIBLE_DEVICES=4 -itd --name=tei-reranker --net=host -e HOME=/home/HwHiAiUser -e LD_PRELOAD= --privileged=true -v /home/BAAI/:/home/HwHiAiUser/model -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver:/usr/local/Ascend/driver --entrypoint /home/HwHiAiUser/start.sh swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.1.RC1-800I-A2-aarch64 BAAI/bge-reranker-large 192.168.99.34 8001
报错日志输出：
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
NPU_ID: 0, CHIP_ID: 0， CHIP_LOGIC_ID: 0 CHIP_TYPE: 910B4, MEMORY_TYPE: HBM, CAPACITY: 32768, USAGE_RATE: 8, AVAIL_CAPACITY: 30146
Using NPU: 0 start TEI service
Starting TEI service on 192.168.99.34:8001...
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
2025-11-18T02:31:51.884008Z INFO text_embeddings_router: router/src/main.rs:189: Args { model_id: "/hom*/**********/*****/***-********-**rge", revision: None, tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 64, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, auto_truncate: false, default_prompt_name: None, default_prompt: None, hf_api_token: None, hf_token: None, hostname: "192.168.99.34", port: 8001, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: None, payload_limit: 2000000, api_key: None, json_output: false, disable_spans: false, otlp_endpoint: None, otlp_service_name: "text-embeddings-inference.server", prometheus_port: 9000, cors_allow_origin: None }
2025-11-18T02:31:52.673300Z WARN text_embeddings_router: router/src/lib.rs:208: Could not find a Sentence Transformers config
2025-11-18T02:31:52.673345Z INFO text_embeddings_router: router/src/lib.rs:212: Maximum number of tokens per request: 512
2025-11-18T02:31:52.673648Z INFO text_embeddings_core::tokenization: core/src/tokenization.rs:38: Starting 192 tokenization workers
2025-11-18T02:32:49.519897Z INFO text_embeddings_router: router/src/lib.rs:254: Starting model backend
2025-11-18T02:32:49.521209Z INFO text_embeddings_backend_python::management: backends/python/src/management.rs:68: Starting Python backend
2025-11-18T02:32:54.684927Z ERROR text_embeddings_backend: backends/src/lib.rs:428: Could not start Python backend: Could not start backend: Python backend failed to start
Error: Could not create backend
Caused by:
Could not start backend: Could not start a suitable backend
2、在哪个步骤出现了问题？
答复：docker启动bge-reranker-large后
3、您希望得到什么结果？
答复：bge-reranker-large能够实现正常运行启动提供服务
4、您实际得到什么结果？
答复：容器启动失败
5、请附上您出现问题页面的整屏截图或者日志信息;
答复：
驱动版本：
然后通过下面的解决方法尝试
通过这个排查，到容器内也同样报错：
完整报错信息：
root@localhost:/home/HwHiAiUser# source /usr/local/Ascend/ascend-toolkit/set_env.sh
root@localhost:/home/HwHiAiUser# source /usr/local/Ascend/nnal/atb/set_env.sh
root@localhost:/home/HwHiAiUser# export LD_PRELOAD=$(ls /usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-*):$LD_PRELOAD
root@localhost:/home/HwHiAiUser# export PATH="/home/HwHiAiUser/.cargo/bin:$PATH"
root@localhost:/home/HwHiAiUser# cat /usr/local/Ascend/
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
cat: /usr/local/Ascend/: Is a directory
root@localhost:/home/HwHiAiUser# ll
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
total 24
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 19 Nov 18 07:13 ./
drwxr-xr-x. 1 root root 24 Jul 29 03:55 ../
-rw-r--r--. 1 HwHiAiUser HwHiAiUser 220 Feb 25 2020 .bash_logout
-rw-r--r--. 1 HwHiAiUser HwHiAiUser 3792 Jul 29 04:00 .bashrc
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 157 Jul 30 09:28 .cargo/
-rw-r--r--. 1 HwHiAiUser HwHiAiUser 828 Jul 29 04:00 .profile
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 73 Jul 30 09:15 .rustup/
drwxr-xr-x. 3 HwHiAiUser HwHiAiUser 40 Jul 30 09:38 Ascend/
drwxr-xr-x. 3 HwHiAiUser HwHiAiUser 19 Jul 30 09:38 log/
drwxr-xr-x 5 root root 4096 Nov 17 08:50 model/
-rwxr-x--x. 1 HwHiAiUser HwHiAiUser 4254 Jun 30 09:07 start.sh*
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 20 Jul 30 10:16 text-embeddings-inference/
root@localhost:/home/HwHiAiUser# if [[ -n $(id | grep uid=0) ]];then
> source /usr/local/Ascend/mxRag/script/set_env.sh
> else
> source /home/HwHiAiUser/Ascend/mxRag/script/set_env.sh
> fi
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
root@localhost:/home/HwHiAiUser# cd /home/HwHiAiUser/text-embeddings-inference/backends/python/server
root@localhost:/home/HwHiAiUser/text-embeddings-inference/backends/python/server# python3 text_embeddings_server/cli.py /home/HwHiAiUser/model/bge-large-zh-v1.5
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
/usr/local/lib/python3.11/dist-packages/torch/distributed/_functional_collectives.py:22: UserWarning: Unable to import torchdynamo util `is_torchdynamo_compiling`, so won't support torchdynamo correctly
warnings.warn(
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1967, in _get_module
return importlib.import_module("." + module_name, self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.11/dist-packages/transformers/integrations/ggml.py", line 24, in <module>
from tokenizers import Tokenizer, decoders, normalizers, pre_tokenizers, processors
File "/usr/local/lib/python3.11/dist-packages/tokenizers/__init__.py", line 94, in <module>
from .implementations import (
File "/usr/local/lib/python3.11/dist-packages/tokenizers/implementations/__init__.py", line 1, in <module>
from .base_tokenizer import BaseTokenizer
File "/usr/local/lib/python3.11/dist-packages/tokenizers/implementations/base_tokenizer.py", line 4, in <module>
from tokenizers.decoders import Decoder
File "/usr/local/lib/python3.11/dist-packages/tokenizers/decoders/__init__.py", line 15, in <module>
DecodeStream = decoders.DecodeStream
^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'decoders' has no attribute 'DecodeStream'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1967, in _get_module
return importlib.import_module("." + module_name, self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.11/dist-packages/transformers/configuration_utils.py", line 27, in <module>
from .modeling_gguf_pytorch_utils import load_gguf_checkpoint
File "/usr/local/lib/python3.11/dist-packages/transformers/modeling_gguf_pytorch_utils.py", line 22, in <module>
from .integrations import (
File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1955, in __getattr__
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1969, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.integrations.ggml because of the following error (look up to see its traceback):
module 'decoders' has no attribute 'DecodeStream'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/cli.py", line 55, in <module>
app()
File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/cli.py", line 42, in serve
from text_embeddings_server import server
File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/server.py", line 3, in <module>
import torch_npu
File "/usr/local/lib/python3.11/dist-packages/torch_npu/__init__.py", line 12, in <module>
from torch.distributed.fsdp import sharded_grad_scaler
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/fsdp/__init__.py", line 1, in <module>
from .flat_param import FlatParameter
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/fsdp/flat_param.py", line 30, in <module>
from torch.distributed.fsdp._common_utils import (
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/fsdp/_common_utils.py", line 31, in <module>
from torch.distributed._tensor.device_mesh import DeviceMesh
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/_tensor/__init__.py", line 319, in <module>
import torch.distributed._tensor._dynamo_utils
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/_tensor/_dynamo_utils.py", line 1, in <module>
from torch._dynamo import allow_in_graph
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/__init__.py", line 2, in <module>
from . import allowed_functions, convert_frame, eval_frame, resume_execution
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/convert_frame.py", line 46, in <module>
from .output_graph import OutputGraph
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/output_graph.py", line 35, in <module>
from . import config, logging as torchdynamo_logging, variables
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/variables/__init__.py", line 53, in <module>
from .torch import TorchVariable
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/variables/torch.py", line 131, in <module>
transformers.configuration_utils.PretrainedConfig.__eq__ = (
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1958, in __getattr__
value = self._get_module(name)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1969, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.configuration_utils because of the following error (look up to see its traceback):
Failed to import transformers.integrations.ggml because of the following error (look up to see its traceback):
module 'decoders' has no attribute 'DecodeStream'
点赞
收藏
回复
分享
举报
已解决
为更快的帮您定位问题，推荐您用以下模板反馈：
1、出现问题时，您做了哪些操作？
答复：
Ascend Docker Runtime最小化验证成功：
参考mis-tei操作的docker指令：docker run -u root -e ASCEND_VISIBLE_DEVICES=4 -itd --name=tei-reranker --net=host -e HOME=/home/HwHiAiUser -e LD_PRELOAD= --privileged=true -v /home/BAAI/:/home/HwHiAiUser/model -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver:/usr/local/Ascend/driver --entrypoint /home/HwHiAiUser/start.sh swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.1.RC1-800I-A2-aarch64 BAAI/bge-reranker-large 192.168.99.34 8001
报错日志输出：
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
NPU_ID: 0, CHIP_ID: 0， CHIP_LOGIC_ID: 0 CHIP_TYPE: 910B4, MEMORY_TYPE: HBM, CAPACITY: 32768, USAGE_RATE: 8, AVAIL_CAPACITY: 30146
Using NPU: 0 start TEI service
Starting TEI service on 192.168.99.34:8001...
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
2025-11-18T02:31:51.884008Z INFO text_embeddings_router: router/src/main.rs:189: Args { model_id: "/hom*/**********/*****/***-********-**rge", revision: None, tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 64, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, auto_truncate: false, default_prompt_name: None, default_prompt: None, hf_api_token: None, hf_token: None, hostname: "192.168.99.34", port: 8001, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: None, payload_limit: 2000000, api_key: None, json_output: false, disable_spans: false, otlp_endpoint: None, otlp_service_name: "text-embeddings-inference.server", prometheus_port: 9000, cors_allow_origin: None }
2025-11-18T02:31:52.673300Z WARN text_embeddings_router: router/src/lib.rs:208: Could not find a Sentence Transformers config
2025-11-18T02:31:52.673345Z INFO text_embeddings_router: router/src/lib.rs:212: Maximum number of tokens per request: 512
2025-11-18T02:31:52.673648Z INFO text_embeddings_core::tokenization: core/src/tokenization.rs:38: Starting 192 tokenization workers
2025-11-18T02:32:49.519897Z INFO text_embeddings_router: router/src/lib.rs:254: Starting model backend
2025-11-18T02:32:49.521209Z INFO text_embeddings_backend_python::management: backends/python/src/management.rs:68: Starting Python backend
2025-11-18T02:32:54.684927Z ERROR text_embeddings_backend: backends/src/lib.rs:428: Could not start Python backend: Could not start backend: Python backend failed to start
Error: Could not create backend
Caused by:
Could not start backend: Could not start a suitable backend
2、在哪个步骤出现了问题？
答复：docker启动bge-reranker-large后
3、您希望得到什么结果？
答复：bge-reranker-large能够实现正常运行启动提供服务
4、您实际得到什么结果？
答复：容器启动失败
5、请附上您出现问题页面的整屏截图或者日志信息;
答复：
驱动版本：
然后通过下面的解决方法尝试
root@localhost:/home/HwHiAiUser# source /usr/local/Ascend/ascend-toolkit/set_env.sh
root@localhost:/home/HwHiAiUser# source /usr/local/Ascend/nnal/atb/set_env.sh
root@localhost:/home/HwHiAiUser# export LD_PRELOAD=$(ls /usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-*):$LD_PRELOAD
root@localhost:/home/HwHiAiUser# export PATH="/home/HwHiAiUser/.cargo/bin:$PATH"
root@localhost:/home/HwHiAiUser# cat /usr/local/Ascend/
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
cat: /usr/local/Ascend/: Is a directory
root@localhost:/home/HwHiAiUser# ll
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
total 24
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 19 Nov 18 07:13 ./
drwxr-xr-x. 1 root root 24 Jul 29 03:55 ../
-rw-r--r--. 1 HwHiAiUser HwHiAiUser 220 Feb 25 2020 .bash_logout
-rw-r--r--. 1 HwHiAiUser HwHiAiUser 3792 Jul 29 04:00 .bashrc
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 157 Jul 30 09:28 .cargo/
-rw-r--r--. 1 HwHiAiUser HwHiAiUser 828 Jul 29 04:00 .profile
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 73 Jul 30 09:15 .rustup/
drwxr-xr-x. 3 HwHiAiUser HwHiAiUser 40 Jul 30 09:38 Ascend/
drwxr-xr-x. 3 HwHiAiUser HwHiAiUser 19 Jul 30 09:38 log/
drwxr-xr-x 5 root root 4096 Nov 17 08:50 model/
-rwxr-x--x. 1 HwHiAiUser HwHiAiUser 4254 Jun 30 09:07 start.sh*
drwxr-xr-x. 1 HwHiAiUser HwHiAiUser 20 Jul 30 10:16 text-embeddings-inference/
root@localhost:/home/HwHiAiUser# if [[ -n $(id | grep uid=0) ]];then
> source /usr/local/Ascend/mxRag/script/set_env.sh
> else
> source /home/HwHiAiUser/Ascend/mxRag/script/set_env.sh
> fi
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
root@localhost:/home/HwHiAiUser# cd /home/HwHiAiUser/text-embeddings-inference/backends/python/server
root@localhost:/home/HwHiAiUser/text-embeddings-inference/backends/python/server# python3 text_embeddings_server/cli.py /home/HwHiAiUser/model/bge-large-zh-v1.5
ERROR: ld.so: object '/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-947d5fa1.so.1.0.0
/usr/local/lib/python3.11/dist-packages/scikit_learn.libs/libgomp-d22c30c5.so.1.0.0' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
/usr/local/lib/python3.11/dist-packages/torch/distributed/_functional_collectives.py:22: UserWarning: Unable to import torchdynamo util `is_torchdynamo_compiling`, so won't support torchdynamo correctly
warnings.warn(
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1967, in _get_module
return importlib.import_module("." + module_name, self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.11/dist-packages/transformers/integrations/ggml.py", line 24, in <module>
from tokenizers import Tokenizer, decoders, normalizers, pre_tokenizers, processors
File "/usr/local/lib/python3.11/dist-packages/tokenizers/__init__.py", line 94, in <module>
from .implementations import (
File "/usr/local/lib/python3.11/dist-packages/tokenizers/implementations/__init__.py", line 1, in <module>
from .base_tokenizer import BaseTokenizer
File "/usr/local/lib/python3.11/dist-packages/tokenizers/implementations/base_tokenizer.py", line 4, in <module>
from tokenizers.decoders import Decoder
File "/usr/local/lib/python3.11/dist-packages/tokenizers/decoders/__init__.py", line 15, in <module>
DecodeStream = decoders.DecodeStream
^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'decoders' has no attribute 'DecodeStream'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1967, in _get_module
return importlib.import_module("." + module_name, self.__name__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.11/dist-packages/transformers/configuration_utils.py", line 27, in <module>
from .modeling_gguf_pytorch_utils import load_gguf_checkpoint
File "/usr/local/lib/python3.11/dist-packages/transformers/modeling_gguf_pytorch_utils.py", line 22, in <module>
from .integrations import (
File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1955, in __getattr__
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1969, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.integrations.ggml because of the following error (look up to see its traceback):
module 'decoders' has no attribute 'DecodeStream'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/cli.py", line 55, in <module>
app()
File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/cli.py", line 42, in serve
from text_embeddings_server import server
File "/home/HwHiAiUser/text-embeddings-inference/backends/python/server/text_embeddings_server/server.py", line 3, in <module>
import torch_npu
File "/usr/local/lib/python3.11/dist-packages/torch_npu/__init__.py", line 12, in <module>
from torch.distributed.fsdp import sharded_grad_scaler
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/fsdp/__init__.py", line 1, in <module>
from .flat_param import FlatParameter
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/fsdp/flat_param.py", line 30, in <module>
from torch.distributed.fsdp._common_utils import (
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/fsdp/_common_utils.py", line 31, in <module>
from torch.distributed._tensor.device_mesh import DeviceMesh
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/_tensor/__init__.py", line 319, in <module>
import torch.distributed._tensor._dynamo_utils
File "/usr/local/lib/python3.11/dist-packages/torch/distributed/_tensor/_dynamo_utils.py", line 1, in <module>
from torch._dynamo import allow_in_graph
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/__init__.py", line 2, in <module>
from . import allowed_functions, convert_frame, eval_frame, resume_execution
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/convert_frame.py", line 46, in <module>
from .output_graph import OutputGraph
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/output_graph.py", line 35, in <module>
from . import config, logging as torchdynamo_logging, variables
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/variables/__init__.py", line 53, in <module>
from .torch import TorchVariable
File "/usr/local/lib/python3.11/dist-packages/torch/_dynamo/variables/torch.py", line 131, in <module>
transformers.configuration_utils.PretrainedConfig.__eq__ = (
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1958, in __getattr__
value = self._get_module(name)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 1969, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.configuration_utils because of the following error (look up to see its traceback):
Failed to import transformers.integrations.ggml because of the following error (look up to see its traceback):
module 'decoders' has no attribute 'DecodeStream'