我分别用mindIE和mis-tei两个docker镜像启动LLM和embedding模型服务,但是遇到一个问题是两个容器只能有一个成功启动模型,第二个容器在启动进程时就报错,报错信息如下:
[ERROR] ASCENDCL(1602,mindieservice_daemon):2025-03-08-10:26:24.218.063 [device.cpp:342]1603 aclrtGetDeviceCount: get device count failed, runtime result = 507899.
Segmentation fault (core dumped)
启动两个容器的方式是:
docker run -e ASCEND_VISIBLE_DEVICES=0 -itd --name=tei --net=host --privileged=true --device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc --device=/dev/davinci0 -v /data/llm/embeddings:/home/HwHiAiUser/model -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver:/usr/local/Ascend/driver embedding:latest BAAI/bge-base-zh-v1.5 192.168.1.8 8026
这个是emebedding的
docker run -itd --name llm --ipc=host --network=host --device=/dev/davinci{4,5,6,7} --device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc -v /usr/local/Ascend/driver:/usr/local/Ascend/driver -v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi -v /usr/local/sbin/:/usr/local/sbin/ -v /var/log/npu/:/usr/slog -v /data/llm/weights:/weights mindie:1.0.0-T71.B020-300I-Duo-arm64-py3.11 bash
2a310787556111fc1f397b9fd1575ac138488f18d8dbbe8e60c2f0b9ad30ce34
这个是大模型的
请问是什么问题?搞了很久搞不定。
[ERROR] ASCENDCL(1602,mindieservice_daemon):2025-03-08-10:26:24.218.063 [device.cpp:342]1603 aclrtGetDeviceCount: get device count failed, runtime result = 507899.
Segmentation fault (core dumped)
启动两个容器的方式是:
docker run -e ASCEND_VISIBLE_DEVICES=0 -itd --name=tei --net=host --privileged=true --device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc --device=/dev/davinci0 -v /data/llm/embeddings:/home/HwHiAiUser/model -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver:/usr/local/Ascend/driver embedding:latest BAAI/bge-base-zh-v1.5 192.168.1.8 8026
这个是emebedding的
docker run -itd --name llm --ipc=host --network=host --device=/dev/davinci{4,5,6,7} --device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc -v /usr/local/Ascend/driver:/usr/local/Ascend/driver -v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi -v /usr/local/sbin/:/usr/local/sbin/ -v /var/log/npu/:/usr/slog -v /data/llm/weights:/weights mindie:1.0.0-T71.B020-300I-Duo-arm64-py3.11 bash
2a310787556111fc1f397b9fd1575ac138488f18d8dbbe8e60c2f0b9ad30ce34
这个是大模型的
请问是什么问题?搞了很久搞不定。