问题:基于mindieservice 多卡推理qwen1.5-7b模型，2卡推理加载失败，4卡推理加载成功，但是推理请求时服务端报错，报错信息和日志如下：
模型加载成功：
推理请求：
报错信息截图：
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Daemon start success!
--- Logging error ---
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
--- Logging error ---
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
其他日志信息截图:
环境如下：
镜像链接：https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48f
点赞
收藏
回复
分享
举报
已解决
问题:基于mindieservice 多卡推理qwen1.5-7b模型，2卡推理加载失败，4卡推理加载成功，但是推理请求时服务端报错，报错信息和日志如下：
模型加载成功：
推理请求：
报错信息截图：
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Daemon start success!
--- Logging error ---
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
--- Logging error ---
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 125, in __generate
request_tokens_np, eof_np = self.__handle_requests(requests, is_prefill)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 190, in __handle_requests
request_tokens_np, eos_np, stop_generate = self.generator.generate_token(metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 66, in generate_token
tokens, eos_np, stop_generate = self.plugin.generate_token(input_metadata)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/plugins/plugin.py", line 12, in generate_token
logits = self.generator_backend.forward(model_inputs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward
logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward
logits = self.forward_tensor(
File "/usr/local/python3.10.2/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor
logits = self.model_runner.forward(
File "/usr/local/Ascend/llm_model/atb_llm/runner/model_runner.py", line 100, in forward
return self.model.forward(**kwargs)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 229, in forward
logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)
File "/usr/local/Ascend/llm_model/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator
acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)
RuntimeError: execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.10.2/lib/python3.10/logging/__init__.py", line 1103, in emit
stream.write(msg + self.terminator)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 85-86: ordinal not in range(128)
Call stack:
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/model.py", line 41, in execute
return self.python_model.execute(requests, config_map)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 81, in execute
return self._prefill(requests)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 95, in _prefill
return self.__generate(requests, is_prefill=True)
File "/usr/local/python3.10.2/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 137, in __generate
logger.exception(f"[Model]\t>>> Exception:{e}")
Message: '[Model]\t>>> Exception:execute\u5931\u8d25 \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'
Arguments: ()
其他日志信息截图:
环境如下：
镜像链接：https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48f