问题:使用mis-tei加载了Qwen3-Embedding-8B模型,加载时配置了多个卡,但是只有第一个卡被使用了,请问如何配置可以使用多个卡?
镜像:mis-tei:7.3.0-300I-Duo-aarch64
启动命令:
docker run -itd --name=mis-tei-7.3.0 --net=host \
--device=/dev/davinci_manager \
--device=/dev/hisi_hdc \
--device=/dev/devmm_svm \
--device=/dev/davinci4:rwm \
--device=/dev/davinci5:rwm \
--device=/dev/davinci6:rwm \
--device=/dev/davinci7:rwm \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
-v /usr/local/sbin:/usr/local/sbin:ro \
-v /AI/llm:/home/HwHiAiUser/model \
-e MAX_BATCH_TOKENS=40960 \
-e PROMETHEUS_PORT=9002 \
swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.3.0-300I-Duo-aarch64 \
Qwen/Qwen3-Embedding-8B 0.0.0.0 8082
镜像:mis-tei:7.3.0-300I-Duo-aarch64
启动命令:
docker run -itd --name=mis-tei-7.3.0 --net=host \
--device=/dev/davinci_manager \
--device=/dev/hisi_hdc \
--device=/dev/devmm_svm \
--device=/dev/davinci4:rwm \
--device=/dev/davinci5:rwm \
--device=/dev/davinci6:rwm \
--device=/dev/davinci7:rwm \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
-v /usr/local/sbin:/usr/local/sbin:ro \
-v /AI/llm:/home/HwHiAiUser/model \
-e MAX_BATCH_TOKENS=40960 \
-e PROMETHEUS_PORT=9002 \
swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.3.0-300I-Duo-aarch64 \
Qwen/Qwen3-Embedding-8B 0.0.0.0 8082