步骤
1、启动容器，将本地的模型挂载到容器中 本地 /data/model 挂载 容器 /models
export IMAGE=swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:2.0.RC2-300I-Duo-py311-openeuler24.03-lts
docker run -d \
--shm-size 500g \
--name qwen-vl \
--device /dev/davinci4 \
--device /dev/davinci5 \
--device /dev/davinci6 \
--device /dev/davinci7 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /data/model:/models:ro \
-p 9025:1025 \
-e VLLM_USE_MODELSCOPE=True \
-e PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256 \
$IMAGE sleep infinity
2、安装atb对应qwen2_vl依赖
3、配置修改
修改模型里面conf.json torch_type 为 float16
修改mindie配置
3、启动mindie服务
cd $MIES_INSTALL_PATH
./bin/mindieservice_daemon
启动日志：
Log default log dir is ~/mindie/log, your can use env MINDIE_LOG_PATH to change log saving dir.
[msservice_profiler] [PID:22719] [DEBUG] [ReadEnable:344] profile enable_: false
[msservice_profiler] [PID:22719] [DEBUG] [ReadAclTaskTime:372] profile enableAclTaskTime_: false
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
Daemon start success!
请求接口 /v1/chat/completions，控制台立即收到报错，如下：
File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2_vl/modeling_qwen2_vl_vit_atb.py", line 937, in forward
vision_features = self.encoder(
^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2_vl/modeling_qwen2_vl_vit_atb.py", line 847, in forward
hidden_states = self.graph.forward(self.graph_inputs, self.graph_outputs, self.graph_param)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The Inner error is reported as above. The process exits for this inner error, and the current working operator name is Qwen2VL_VIT_graph.
Since the operator is called asynchronously, the stacktrace may be inaccurate. If you want to get the accurate stacktrace, pleace set the environment variable ASCEND_LAUNCH_BLOCKING=1.
Note: ASCEND_LAUNCH_BLOCKING=1 will force ops to run in synchronous mode, resulting in performance degradation. Please unset ASCEND_LAUNCH_BLOCKING in time after debugging.
[ERROR] 2025-07-03-15:49:28 (PID:22828, Device:1, RankID:-1) ERR00100 PTA call acl api failed.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib64/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.11/threading.py", line 982, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 163, in do_inference
responses = self.execute(requests, config_map)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 74, in execute
return self.python_model.execute(requests, config_map)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 187, in execute
_print_component_error_log(e)
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 55, in _print_component_error_log
raise RuntimeError(f"CANN execute error, Exception: {e}") from e
RuntimeError: CANN execute error, Exception: The Inner error is reported as above. The process exits for this inner error, and the current working operator name is Qwen2VL_VIT_graph.
Since the operator is called asynchronously, the stacktrace may be inaccurate. If you want to get the accurate stacktrace, pleace set the environment variable ASCEND_LAUNCH_BLOCKING=1.
Note: ASCEND_LAUNCH_BLOCKING=1 will force ops to run in synchronous mode, resulting in performance degradation. Please unset ASCEND_LAUNCH_BLOCKING in time after debugging.
[ERROR] 2025-07-03-15:49:28 (PID:22828, Device:1, RankID:-1) ERR00100 PTA call acl api failed.
点赞
收藏
回复
分享
举报
已解决
步骤
1、启动容器，将本地的模型挂载到容器中 本地 /data/model 挂载 容器 /models
export IMAGE=swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:2.0.RC2-300I-Duo-py311-openeuler24.03-lts
docker run -d \
--shm-size 500g \
--name qwen-vl \
--device /dev/davinci4 \
--device /dev/davinci5 \
--device /dev/davinci6 \
--device /dev/davinci7 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /data/model:/models:ro \
-p 9025:1025 \
-e VLLM_USE_MODELSCOPE=True \
-e PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256 \
$IMAGE sleep infinity
2、安装atb对应qwen2_vl依赖
3、配置修改
修改模型里面conf.json torch_type 为 float16
修改mindie配置
3、启动mindie服务
cd $MIES_INSTALL_PATH
./bin/mindieservice_daemon
启动日志：
Log default log dir is ~/mindie/log, your can use env MINDIE_LOG_PATH to change log saving dir.
[msservice_profiler] [PID:22719] [DEBUG] [ReadEnable:344] profile enable_: false
[msservice_profiler] [PID:22719] [DEBUG] [ReadAclTaskTime:372] profile enableAclTaskTime_: false
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
Daemon start success!
请求接口 /v1/chat/completions，控制台立即收到报错，如下：
File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2_vl/modeling_qwen2_vl_vit_atb.py", line 937, in forward
vision_features = self.encoder(
^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/Ascend/atb-models/atb_llm/models/qwen2_vl/modeling_qwen2_vl_vit_atb.py", line 847, in forward
hidden_states = self.graph.forward(self.graph_inputs, self.graph_outputs, self.graph_param)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The Inner error is reported as above. The process exits for this inner error, and the current working operator name is Qwen2VL_VIT_graph.
Since the operator is called asynchronously, the stacktrace may be inaccurate. If you want to get the accurate stacktrace, pleace set the environment variable ASCEND_LAUNCH_BLOCKING=1.
Note: ASCEND_LAUNCH_BLOCKING=1 will force ops to run in synchronous mode, resulting in performance degradation. Please unset ASCEND_LAUNCH_BLOCKING in time after debugging.
[ERROR] 2025-07-03-15:49:28 (PID:22828, Device:1, RankID:-1) ERR00100 PTA call acl api failed.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib64/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.11/threading.py", line 982, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 163, in do_inference
responses = self.execute(requests, config_map)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 74, in execute
return self.python_model.execute(requests, config_map)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 187, in execute
_print_component_error_log(e)
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 55, in _print_component_error_log
raise RuntimeError(f"CANN execute error, Exception: {e}") from e
RuntimeError: CANN execute error, Exception: The Inner error is reported as above. The process exits for this inner error, and the current working operator name is Qwen2VL_VIT_graph.
Since the operator is called asynchronously, the stacktrace may be inaccurate. If you want to get the accurate stacktrace, pleace set the environment variable ASCEND_LAUNCH_BLOCKING=1.
Note: ASCEND_LAUNCH_BLOCKING=1 will force ops to run in synchronous mode, resulting in performance degradation. Please unset ASCEND_LAUNCH_BLOCKING in time after debugging.
[ERROR] 2025-07-03-15:49:28 (PID:22828, Device:1, RankID:-1) ERR00100 PTA call acl api failed.