使用镜像：swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts
已按照https://gitee.com/ascend/ModelZoo-PyTorch/tree/master/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-14B 中的操作步骤，使用一张300IDUO的卡完成了：
1、稀疏量化、量化权重切分及压缩
python3 quant_qwen.py --model_path /root/llm/DeepSeek-R1-Distill-Qwen-14B/ --save_directory /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8S/ --calib_file ../common/cn_en.jsonl --w_bit 4 --a_bit 8 --fraction 0.011 --co_sparse True --device_type npu --use_sigma True --is_lowbit True --sigma_factor 4.0 --anti_method m4
torchrun --nproc_per_node 2 -m examples.convert.model_slim.sparse_compressor --multiprocess_num 4 --model_path /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8S --save_directory /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8SC
2、对话测试
torchrun --nproc_per_node 2 --master_port 20037 -m examples.run_pa --model_path /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8SC --max_output_length 20
上述操作都正常，但是进行服务化推理的时候报错：
2025-04-22 13:58:12,977 [ERROR] model.py:39 - [Model] >>> Exception:npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116649, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116649] 2025-04-22-13:58:12.850.127 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=0, stream_id=4, report_stream_id=4, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[116649]:
EXCEPTION STREAM:
Exception info:TGID=3113516, model id=65535, stream id=4, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=1, stream_id=4
Other info[0]:time=2025-04-22-13:58:12.689.847, function=process_hwts_error_exception, line=975, error code=0x26
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 37, in initialize
return self.python_model.initialize(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 146, in initialize
self.generator = Generator(
^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 119, in __init__
self.warm_up(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times, inference_mode)
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 305, in warm_up
npu.synchronize()
File "/usr/local/lib64/python3.11/site-packages/torch_npu/npu/utils.py", line 35, in synchronize
return torch_npu._C._npu_synchronize()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116649, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116649] 2025-04-22-13:58:12.850.127 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=0, stream_id=4, report_stream_id=4, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[116649]:
EXCEPTION STREAM:
Exception info:TGID=3113516, model id=65535, stream id=4, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=1, stream_id=4
Other info[0]:time=2025-04-22-13:58:12.689.847, function=process_hwts_error_exception, line=975, error code=0x26
2025-04-22 13:58:12,978 [ERROR] model.py:42 - [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0'}
2025-04-22 13:58:12,979 [ERROR] model.py:39 - [Model] >>> Exception:npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116651, Device:1, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116651] 2025-04-22-13:58:12.851.234 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=1, stream_id=2, report_stream_id=2, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[1] PID[116651]:
EXCEPTION STREAM:
Exception info:TGID=3113518, model id=65535, stream id=2, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=29, stream_id=2
Other info[0]:time=2025-04-22-13:58:12.690.571, function=process_hwts_error_exception, line=975, error code=0x26
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 37, in initialize
return self.python_model.initialize(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 146, in initialize
self.generator = Generator(
^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 119, in __init__
self.warm_up(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times, inference_mode)
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 305, in warm_up
npu.synchronize()
File "/usr/local/lib64/python3.11/site-packages/torch_npu/npu/utils.py", line 35, in synchronize
return torch_npu._C._npu_synchronize()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116651, Device:1, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116651] 2025-04-22-13:58:12.851.234 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=1, stream_id=2, report_stream_id=2, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[1] PID[116651]:
EXCEPTION STREAM:
Exception info:TGID=3113518, model id=65535, stream id=2, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=29, stream_id=2
Other info[0]:time=2025-04-22-13:58:12.690.571, function=process_hwts_error_exception, line=975, error code=0x26
2025-04-22 13:58:12,981 [ERROR] model.py:42 - [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0'}
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
对于14B量化未切分模型的服务化推理可正常运行，但切分后会出现以上错误。
点赞
收藏
回复
分享
举报
已解决
使用镜像：swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts
已按照https://gitee.com/ascend/ModelZoo-PyTorch/tree/master/MindIE/LLM/DeepSeek/DeepSeek-R1-Distill-Qwen-14B 中的操作步骤，使用一张300IDUO的卡完成了：
1、稀疏量化、量化权重切分及压缩
python3 quant_qwen.py --model_path /root/llm/DeepSeek-R1-Distill-Qwen-14B/ --save_directory /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8S/ --calib_file ../common/cn_en.jsonl --w_bit 4 --a_bit 8 --fraction 0.011 --co_sparse True --device_type npu --use_sigma True --is_lowbit True --sigma_factor 4.0 --anti_method m4
torchrun --nproc_per_node 2 -m examples.convert.model_slim.sparse_compressor --multiprocess_num 4 --model_path /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8S --save_directory /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8SC
2、对话测试
torchrun --nproc_per_node 2 --master_port 20037 -m examples.run_pa --model_path /root/llm/DeepSeek-R1-Distill-Qwen-14B-W8A8SC --max_output_length 20
上述操作都正常，但是进行服务化推理的时候报错：
2025-04-22 13:58:12,977 [ERROR] model.py:39 - [Model] >>> Exception:npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116649, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116649] 2025-04-22-13:58:12.850.127 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=0, stream_id=4, report_stream_id=4, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[116649]:
EXCEPTION STREAM:
Exception info:TGID=3113516, model id=65535, stream id=4, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=1, stream_id=4
Other info[0]:time=2025-04-22-13:58:12.689.847, function=process_hwts_error_exception, line=975, error code=0x26
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 37, in initialize
return self.python_model.initialize(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 146, in initialize
self.generator = Generator(
^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 119, in __init__
self.warm_up(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times, inference_mode)
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 305, in warm_up
npu.synchronize()
File "/usr/local/lib64/python3.11/site-packages/torch_npu/npu/utils.py", line 35, in synchronize
return torch_npu._C._npu_synchronize()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116649, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116649] 2025-04-22-13:58:12.850.127 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=0, stream_id=4, report_stream_id=4, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[0] PID[116649]:
EXCEPTION STREAM:
Exception info:TGID=3113516, model id=65535, stream id=4, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=1, stream_id=4
Other info[0]:time=2025-04-22-13:58:12.689.847, function=process_hwts_error_exception, line=975, error code=0x26
2025-04-22 13:58:12,978 [ERROR] model.py:42 - [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0'}
2025-04-22 13:58:12,979 [ERROR] model.py:39 - [Model] >>> Exception:npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116651, Device:1, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116651] 2025-04-22-13:58:12.851.234 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=1, stream_id=2, report_stream_id=2, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[1] PID[116651]:
EXCEPTION STREAM:
Exception info:TGID=3113518, model id=65535, stream id=2, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=29, stream_id=2
Other info[0]:time=2025-04-22-13:58:12.690.571, function=process_hwts_error_exception, line=975, error code=0x26
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 37, in initialize
return self.python_model.initialize(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 146, in initialize
self.generator = Generator(
^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 119, in __init__
self.warm_up(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times, inference_mode)
File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 305, in warm_up
npu.synchronize()
File "/usr/local/lib64/python3.11/site-packages/torch_npu/npu/utils.py", line 35, in synchronize
return torch_npu._C._npu_synchronize()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: npuSynchronizeDevice:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:467 NPU function error: AclrtSynchronizeDeviceWithTimeout, error code is 507015
[ERROR] 2025-04-22-13:58:12 (PID:116651, Device:1, RankID:-1) ERR00100 PTA call acl api failed
[Error]: The aicore execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 116651] 2025-04-22-13:58:12.851.234 Kernel task happen error, retCode=0x26, [aicore exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1356]
TraceBack (most recent call last):
Aicore kernel execute failed, device_id=1, stream_id=2, report_stream_id=2, task_id=2525, flip_num=0, fault kernel_name=pp_matmul_int8_nz_compress_0, fault kernel info ext=pp_matmul_int8_nz_compress_0, program id=87, hash=3087916139699476489.[FUNC:GetError][FILE:stream.cc][LINE:1124]
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1124]
rtDeviceSynchronizeWithTimeout execute failed, reason=[aicore exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507015.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
DEVICE[1] PID[116651]:
EXCEPTION STREAM:
Exception info:TGID=3113518, model id=65535, stream id=2, stream phase=3
Message info[0]:RTS_HWTS: Aicore exception, slot_id=29, stream_id=2
Other info[0]:time=2025-04-22-13:58:12.690.571, function=process_hwts_error_exception, line=975, error code=0x26
2025-04-22 13:58:12,981 [ERROR] model.py:42 - [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0'}
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!
对于14B量化未切分模型的服务化推理可正常运行，但切分后会出现以上错误。