昇腾论坛

环境信息：

os：Linux kuntai 4.19.90-52.22.v2207.ky10.aarch64 #1 SMP Tue Mar 14 11:52:45 CST 2023 aarch64 aarch64 aarch64 GNU/Linux

GPU：300i duo（Device d500）

python：Python 3.10.14

驱动：24.1.rc2

cann：8.0.RC2

mindie：1.0.RC2

1. 用atb自带的工具转换：python quant_qwen2_14b_w4a16_64.py /data/models/Qwen1.5-14B-Chat /data/models/Qwen1.5-14B-Chat-W4A16

2. 用mindie启动报错：

2024-10-10 18:26:10,419 [INFO] [pid: 1171824] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': '0,1,2,3', 'use_host_chooser': True, 'bind_cpu': True}

2024-10-10 18:26:10,589 [INFO] [pid: 1171822] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': '0,1,2,3', 'use_host_chooser': True, 'bind_cpu': True}

2024-10-10 18:26:10,596 [INFO] [pid: 1171825] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': '0,1,2,3', 'use_host_chooser': True, 'bind_cpu': True}

2024-10-10 18:26:10,713 [INFO] [pid: 1171823] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': '0,1,2,3', 'use_host_chooser': True, 'bind_cpu': True}

2024-10-10 18:26:13,634 [INFO] [pid: 1171824] cpu_binding.py-206: rank_id: 2, device_id: 2, numa_id: 2, shard_devices: [2, 3], cpus: [48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71]

2024-10-10 18:26:13,636 [INFO] [pid: 1171824] cpu_binding.py-231: process 1171824, new_affinity is [48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59], cpu count 12

The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.

2024-10-10 18:26:14,172 [INFO] [pid: 1171825] cpu_binding.py-206: rank_id: 3, device_id: 3, numa_id: 2, shard_devices: [2, 3], cpus: [48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71]

2024-10-10 18:26:14,173 [INFO] [pid: 1171825] cpu_binding.py-231: process 1171825, new_affinity is [60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71], cpu count 12

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

2024-10-10 18:26:14,202 [INFO] [pid: 1171822] cpu_binding.py-206: rank_id: 0, device_id: 0, numa_id: 0, shard_devices: [0, 1], cpus: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]

2024-10-10 18:26:14,204 [INFO] [pid: 1171822] cpu_binding.py-231: process 1171822, new_affinity is [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], cpu count 12

2024-10-10 18:26:14,274 [INFO] [pid: 1171823] cpu_binding.py-206: rank_id: 1, device_id: 1, numa_id: 0, shard_devices: [0, 1], cpus: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]

2024-10-10 18:26:14,275 [INFO] [pid: 1171823] cpu_binding.py-231: process 1171823, new_affinity is [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23], cpu count 12

The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.

[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

2024-10-10 18:26:14,763 [INFO] [pid: 1171822] logging.py-53: model_runner.quantize: w4a16

, model_runner.kv_quant: None

, model_runner.dytpe: torch.float16

2024-10-10 18:26:14,763 [INFO] [pid: 1171822] logging.py-53: Rank table file location:

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())

2024-10-10 18:26:20,332 [INFO] [pid: 1171822] dist.py-94: initialize_distributed has been Set

2024-10-10 18:26:20,334 [INFO] [pid: 1171822] logging.py-53: init tokenizer done: Qwen2TokenizerFast(name_or_path='/data/models/Qwen1.5-14B-Chat-W4A16', vocab_size=151643, model_max_length=32768, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>']}, clean_up_tokenization_spaces=False), added_tokens_decoder={

151643: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),

151644: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),

151645: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),

}

2024-10-10 18:26:20,459 [INFO] [pid: 1171822] logging.py-53: NPUSocInfo(soc_name='', soc_version=202, need_nz=True)

2024-10-10 18:26:20,488 [INFO] [pid: 1171822] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.

2024-10-10 18:26:20,691 [INFO] [pid: 1171825] dist.py-94: initialize_distributed has been Set

2024-10-10 18:26:20,882 [INFO] [pid: 1171825] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.

2024-10-10 18:26:20,926 [INFO] [pid: 1171823] dist.py-94: initialize_distributed has been Set

2024-10-10 18:26:21,175 [INFO] [pid: 1171823] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.

2024-10-10 18:26:24,345 [INFO] [pid: 1171824] dist.py-94: initialize_distributed has been Set

2024-10-10 18:26:24,764 [INFO] [pid: 1171824] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.

2024-10-10 18:26:36,486 [INFO] [pid: 1171822] logging.py-53: model:

FlashQwen2ForCausalLM(

(rotary_embedding): PositionRotaryEmbedding()

(attn_mask): AttentionMask()

(transformer): FlashQwenModel(

(wte): TensorParallelEmbedding()

(h): ModuleList(

(0-19): 20 x FlashQwenLayer(

(attn): FlashQwenAttention(

(rotary_emb): PositionRotaryEmbedding()

(c_attn): TensorParallelColumnLinear(

(linear): W4A16LinearStatic()

)

(c_proj): TensorParallelRowLinear(

(linear): W4A16LinearStatic()

)

(mlp): QwenMLP(

(act): SiLU()

(w2_w1): TensorParallelColumnLinear(

(linear): W4A16LinearStatic()

)

(c_proj): TensorParallelRowLinear(

(linear): FastLinear()

)

(ln_1): QwenRMSNorm()

(ln_2): QwenRMSNorm()

)

(20-39): 20 x FlashQwenLayer(

(attn): FlashQwenAttention(

(rotary_emb): PositionRotaryEmbedding()

(c_attn): TensorParallelColumnLinear(

(linear): W4A16LinearStatic()

)

(c_proj): TensorParallelRowLinear(

(linear): W4A16LinearStatic()

)

(mlp): QwenMLP(

(act): SiLU()

(w2_w1): TensorParallelColumnLinear(

(linear): W4A16LinearStatic()

)

(c_proj): TensorParallelRowLinear(

(linear): W4A16LinearStatic()

)

(ln_1): QwenRMSNorm()

(ln_2): QwenRMSNorm()

)

(ln_f): QwenRMSNorm()

)

(lm_head): TensorParallelHead(

(linear): FastLinear()

)

2024-10-10 18:26:36,530 [INFO] [pid: 1171823] flash_causal_lm.py-127: <<<<<<<after transdata k_caches[0].shape=torch.Size([327, 80, 128, 16])

2024-10-10 18:26:36,615 [INFO] [pid: 1171825] flash_causal_lm.py-127: <<<<<<<after transdata k_caches[0].shape=torch.Size([327, 80, 128, 16])

2024-10-10 18:26:36,633 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,634 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,634 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,634 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,635 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,635 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,635 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,636 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,636 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,636 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,637 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,637 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,637 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,638 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,638 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,638 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,639 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,639 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,639 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,640 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,640 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,640 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,641 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,641 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,641 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,642 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,642 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,642 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,643 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,643 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,643 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,644 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,644 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,644 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,645 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,645 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,645 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,646 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,646 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,646 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,646 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,647 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,647 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,648 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,648 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,648 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,649 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,649 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,649 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,650 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,651 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,652 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,653 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,654 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,655 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,656 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,657 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,658 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,659 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,660 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,661 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,662 [INFO] [pid: 1171822] logging.py-53: trans to 2

2024-10-10 18:26:36,664 [INFO] [pid: 1171822] logging.py-53: trans to 29

2024-10-10 18:26:36,699 [INFO] [pid: 1171822] logging.py-53: <<<<<<< ori k_caches[0].shape=torch.Size([327, 80, 128, 16])

2024-10-10 18:26:36,701 [INFO] [pid: 1171822] flash_causal_lm.py-127: <<<<<<<after transdata k_caches[0].shape=torch.Size([327, 80, 128, 16])

2024-10-10 18:26:36,701 [INFO] [pid: 1171822] logging.py-53: >>>>>>id of kcache is 281458018667152 id of vcache is 281458018667072

2024-10-10 18:26:39,200 [INFO] [pid: 1171824] flash_causal_lm.py-127: <<<<<<<after transdata k_caches[0].shape=torch.Size([327, 80, 128, 16])

--- Logging error ---

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 28, in initialize

return self.python_model.initialize(config)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 28, in initialize

self.generator = Generator(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 56, in __init__

self.warm_up(max_prefill_tokens, max_seq_len, max_iter_times)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 133, in warm_up

self.generator_backend.warm_up(model_inputs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 42, in warm_up

_ = self.forward(model_inputs, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward

logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward

logits = self.forward_tensor(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor

logits = self.model_runner.forward(

File "/usr/local/Ascend/atb/atb_llm/runner/model_runner.py", line 100, in forward

return self.model.forward(**kwargs)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 229, in forward

logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator

acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)

RuntimeError: Setup\u65f6\u5931\u8d25\uff0cError Code: 12, \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/logging/__init__.py", line 1103, in emit

stream.write(msg + self.terminator)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-76: ordinal not in range(128)

Call stack:

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 30, in initialize

logger.exception(f"[Model]\t>>> Exception:{e}")

Message: '[Model]\t>>> Exception:Setup\u65f6\u5931\u8d25\uff0cError Code: 12, \u8be6\u7ec6\u4fe1\u606f\u89c1Ascend\u5b98\u65b9\u6587\u6863\uff0c\u8bf7\u5f00\u542f\u65e5\u5fd7\u8fdb\u4e00\u6b65\u5b9a\u4f4d\u95ee\u9898\u3002\n'

Arguments: ()

--- Logging error ---

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 28, in initialize

return self.python_model.initialize(config)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 28, in initialize

self.generator = Generator(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 56, in __init__

self.warm_up(max_prefill_tokens, max_seq_len, max_iter_times)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 133, in warm_up

self.generator_backend.warm_up(model_inputs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 42, in warm_up

_ = self.forward(model_inputs, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward

logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward

logits = self.forward_tensor(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor

logits = self.model_runner.forward(

File "/usr/local/Ascend/atb/atb_llm/runner/model_runner.py", line 100, in forward

return self.model.forward(**kwargs)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 229, in forward

logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator

acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/logging/__init__.py", line 1103, in emit

stream.write(msg + self.terminator)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-76: ordinal not in range(128)

Call stack:

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 30, in initialize

logger.exception(f"[Model]\t>>> Exception:{e}")

Arguments: ()

--- Logging error ---

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 28, in initialize

return self.python_model.initialize(config)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 28, in initialize

self.generator = Generator(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 56, in __init__

self.warm_up(max_prefill_tokens, max_seq_len, max_iter_times)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 133, in warm_up

self.generator_backend.warm_up(model_inputs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 42, in warm_up

_ = self.forward(model_inputs, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward

logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward

logits = self.forward_tensor(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor

logits = self.model_runner.forward(

File "/usr/local/Ascend/atb/atb_llm/runner/model_runner.py", line 100, in forward

return self.model.forward(**kwargs)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 229, in forward

logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator

acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/logging/__init__.py", line 1103, in emit

stream.write(msg + self.terminator)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-76: ordinal not in range(128)

Call stack:

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 30, in initialize

logger.exception(f"[Model]\t>>> Exception:{e}")

Arguments: ()

--- Logging error ---

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 28, in initialize

return self.python_model.initialize(config)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/standard_model.py", line 28, in initialize

self.generator = Generator(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 56, in __init__

self.warm_up(max_prefill_tokens, max_seq_len, max_iter_times)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/generator.py", line 133, in warm_up

self.generator_backend.warm_up(model_inputs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 42, in warm_up

_ = self.forward(model_inputs, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 52, in forward

logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 60, in forward

logits = self.forward_tensor(

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 86, in forward_tensor

logits = self.model_runner.forward(

File "/usr/local/Ascend/atb/atb_llm/runner/model_runner.py", line 100, in forward

return self.model.forward(**kwargs)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 229, in forward

logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill)

File "/usr/local/Ascend/atb/atb_llm/models/base/flash_causal_lm.py", line 200, in execute_ascend_operator

acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/logging/__init__.py", line 1103, in emit

stream.write(msg + self.terminator)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-76: ordinal not in range(128)

Call stack:

File "/data/miniconda3/envs/test/lib/python3.10/site-packages/model_wrapper/model.py", line 30, in initialize

logger.exception(f"[Model]\t>>> Exception:{e}")

Arguments: ()

Process ForkServerProcess-7:

Process ForkServerProcess-5:

Process ForkServerProcess-2:

Process ForkServerProcess-6:

Process ForkServerProcess-4:

Traceback (most recent call last):

Process ForkServerProcess-3:

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

Traceback (most recent call last):

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

Process ForkServerProcess-8:

Traceback (most recent call last):

Process ForkServerProcess-9:

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

File "<string>", line 2, in get

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

File "<string>", line 2, in get

EOFError

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

EOFError

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

File "<string>", line 2, in get

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

EOFError

Traceback (most recent call last):

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

File "<string>", line 2, in get

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

self.run()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/process.py", line 108, in run

self._target(*self._args, **self._kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 65, in wrapper

raise exp

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

File "<string>", line 2, in get

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

EOFError

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 62, in wrapper

func(*args, **kwargs)

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

File "<string>", line 2, in get

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

EOFError

File "/usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/common/repository_manager/route.py", line 262, in task_distribute

key, func_name, detail = resource_proxy[TASK_QUEUE].get()

EOFError

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

File "<string>", line 2, in get

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/managers.py", line 818, in _callmethod

kind, result = conn.recv()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 250, in recv

buf = self._recv_bytes()

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes

buf = self._recv(4)

File "/data/miniconda3/envs/test/lib/python3.10/multiprocessing/connection.py", line 383, in _recv

raise EOFError

EOFError

Daemon wait pid with 1171824, status 139Daemon wait pid with 1171823, status 139Daemon is killing...

关于昇腾

新闻与活动

交流与资讯

支持与服务

开源社区