Loading weights from safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.15it/s] Process mp_engine_proc: Traceback (most recent call last): File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/local/python3.11.13/lib/python3.11/site-packages/lmdeploy/pytorch/engine/mp_engine/zmq_engine.py", line 109, in _mp_proc engine = Engine.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/python3.11.13/lib/python3.11/site-packages/lmdeploy/pytorch/engine/engine.py", line 225, in from_pretrained return cls( ^^^^ File "/usr/local/python3.11.13/lib/python3.11/site-packages/lmdeploy/pytorch/engine/engine.py", line 145, in init self.executor.init() File "/usr/local/python3.11.13/lib/python3.11/site-packages/lmdeploy/pytorch/engine/executor/base.py", line 240, in init self.build_model() File "/usr/local/python3.11.13/lib/python3.11/site-packages/lmdeploy/pytorch/engine/executor/uni_executor.py", line 60, in build_model self.model_agent.build_model() File "/usr/local/python3.11.13/lib/python3.11/site-packages/lmdeploy/pytorch/engine/model_agent.py", line 1042, in build_model self._build_model() File "/usr/local/python3.11.13/lib/python3.11/site-packages/dlinfer/framework/lmdeploy_ext/device/ascend.py", line 520, in _build_model_310P load_model_weights_310P(patched_model, model_path, device=device) File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/python3.11.13/lib/python3.11/site-packages/dlinfer/framework/lmdeploy_ext/device/ascend.py", line 487, in load_model_weights_310P loader.load_model_weights(model, device=device) File "/usr/local/python3.11.13/lib/python3.11/site-packages/lmdeploy/pytorch/weight_loader/model_weight_loader.py", line 189, in load_model_weights model.to(device) File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py", line 182, in decorated return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/_module.py", line 70, in to self.cast_weight(device) File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/_module.py", line 140, in cast_weight sub_module.cast_weight(device) File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/_module.py", line 140, in cast_weight sub_module.cast_weight(device) File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/_module.py", line 140, in cast_weight sub_module.cast_weight(device) File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/_module.py", line 133, in cast_weight _format_cast(self, current_class) File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/utils/_module.py", line 118, in _format_cast module.weight.data = torch_npu.npu_format_cast(module.weight.data, 4) # ACL_FORMAT_FRACTAL_Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/_ops.py", line 1243, in call return self._op(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: InnerRun:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:240 OPS function error: Identity, error code is 500002 [ERROR] 2026-01-14-07:01:49 (PID:64055, Device:0, RankID:-1) ERR01100 OPS call acl api failed [Error]: A GE error occurs in the system. Rectify the fault based on the error information in the ascend log. [PID: 64055] 2026-01-14-07:01:49.465.130 Unsupported_Operator(EZ3002): Optype [TransData] of Ops kernel [AIcoreEngine] is unsupported. Reason: [tbe-custom]:op type TransData is not found in this op store.[tbe-custom]:op type TransData is not found in this op store.[Dynamic shape check]: The format and dtype is not precisely equivalent to format and dtype in op information library[Static shape check]:The format and dtype is not precisely equivalent to format and dtype in op information library. Possible Cause: The operator type is unsupported in the operator information library due to specification mismatch. Solution: Submit an issue to request for support at https://gitee.com/ascend, or remove this type of operators from your model. TraceBack (most recent call last): Optype [TransData] of Ops kernel [DNN_VM_HOST_CPU_OP_STORE] is unsupported. Reason: Transdata op, groups should be greater than 1, but now is 1. Optype [TransData] of Ops kernel [aicpu_ascend_kernel] is unsupported. Reason: Transdata op, groups should be greater than 1, but now is 1. No supported Ops kernel and engine are found for [trans_TransData_50], optype [TransData]. Failed to select engine for [trans_TransData_50][TransData].[FUNC:operator()][FILE:engine_place.cc][LINE:150] RunAllSubgraphs failed, graph=online.[FUNC:RunAllSubgraphs][FILE:engine_place.cc][LINE:123] build graph failed, graph id:29, ret:4294967295[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1594] [Build][SingleOpModel]call ge interface generator.BuildSingleOpModel failed. ge result = 4294967295[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162] [Build][Op]Fail to build op model[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:146] build op model failed, result = 500002[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:146]

/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py:330: ResourceWarning: Unclosed socket <zmq.Socket(zmq.ROUTER) at 0xfffeba1f3850> traceback.print_exc() /usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py:330: ResourceWarning: Unclosed context <zmq.Context() at 0xfffeb9fa9a90> traceback.print_exc() ^C[ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared! [ERROR] TBE Subprocess[task_distribute] raise error[], main process disappeared!