我使用310P3部署Qwen3-Embedding-0.6B
npu信息:
npu-smi info
+--------------------------------------------------------------------------------------------------------+
| npu-smi 25.0.rc1.1 Version: 25.0.rc1.1 |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+===============================+=================+======================================================+
| 1536 310P3 | OK | NA 40 0 / 0 |
| 0 0 | 0000:06:00.0 | 0 1344 / 44280 |
+-------------------------------+-----------------+------------------------------------------------------+
| 1536 310P3 | OK | NA 40 0 / 0 |
| 1 1 | 0000:06:00.0 | 0 1592 / 43693 |
+===============================+=================+======================================================+
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===============================+=================+======================================================+ |
使用的TEI是最新的:swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.2.RC1-300I-Duo-aarch64
docker run -itd \
--name qwen3-embedding \
--privileged \
--network host \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci_manager \
--device /dev/mm_svm \
--device /dev/hisi_hdc \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /home/HwHiAiUser/model/Qwen3-Embedding-0.6B:/home/HwHiAiUser/model/Qwen3-Embedding-0.6B \
-e MODEL_ID=/home/HwHiAiUser/model/ \
-e TRANSFORMERS_MODEL_NAME=/home/HwHiAiUser/model \
swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.2.RC1-300I-Duo-aarch64 \
Qwen/Qwen3-Embedding-0.6B 127.0.0.1 9990
启动后报错:
Error: `--max-batch-tokens` cannot be lower than the model `max_input_length` (32768 < 16384) when `--auto-truncate` is disabled, add the `--auto-truncate` flag to truncate the input sequences to match the `--max-batch-tokens`.
这应该怎么解决。我之前在910B3 用 swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.1.RC1-800I-A2-aarch64,也是一样的启动方式没问题,不报错。
我使用310P3部署Qwen3-Embedding-0.6B
npu信息:
npu-smi info
+--------------------------------------------------------------------------------------------------------+
| npu-smi 25.0.rc1.1 Version: 25.0.rc1.1 |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+===============================+=================+======================================================+
| 1536 310P3 | OK | NA 40 0 / 0 |
| 0 0 | 0000:06:00.0 | 0 1344 / 44280 |
+-------------------------------+-----------------+------------------------------------------------------+
| 1536 310P3 | OK | NA 40 0 / 0 |
| 1 1 | 0000:06:00.0 | 0 1592 / 43693 |
+===============================+=================+======================================================+
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===============================+=================+======================================================+ |
使用的TEI是最新的:swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.2.RC1-300I-Duo-aarch64docker run -itd \
--name qwen3-embedding \
--privileged \
--network host \
--device /dev/davinci0 \
--device /dev/davinci1 \
--device /dev/davinci_manager \
--device /dev/mm_svm \
--device /dev/hisi_hdc \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /home/HwHiAiUser/model/Qwen3-Embedding-0.6B:/home/HwHiAiUser/model/Qwen3-Embedding-0.6B \
-e MODEL_ID=/home/HwHiAiUser/model/ \
-e TRANSFORMERS_MODEL_NAME=/home/HwHiAiUser/model \
swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.2.RC1-300I-Duo-aarch64 \
Qwen/Qwen3-Embedding-0.6B 127.0.0.1 9990
启动后报错:
Error: `--max-batch-tokens` cannot be lower than the model `max_input_length` (32768 < 16384) when `--auto-truncate` is disabled, add the `--auto-truncate` flag to truncate the input sequences to match the `--max-batch-tokens`.
这应该怎么解决。我之前在910B3 用 swr.cn-south-1.myhuaweicloud.com/ascendhub/mis-tei:7.1.RC1-800I-A2-aarch64,也是一样的启动方式没问题,不报错。