Daemon wait pid with 1595, status 02025-01-07 09:31:30,947 [INFO] [pid: 1607] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': None, 'use_host_chooser': True, 'bind_cpu': True}
2025-01-07 09:31:30,948 [INFO] [pid: 1608] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': None, 'use_host_chooser': True, 'bind_cpu': True}
2025-01-07 09:31:30,969 [INFO] [pid: 1610] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': None, 'use_host_chooser': True, 'bind_cpu': True}
2025-01-07 09:31:30,969 [INFO] [pid: 1609] env.py-55: {'use_ascend': True, 'max_memory_gb': None, 'reserved_memory_gb': 3, 'skip_warmup': False, 'visible_devices': None, 'use_host_chooser': True, 'bind_cpu': True}
2025-01-07 09:31:33,411 [INFO] [pid: 1607] cpu_binding.py-206: rank_id: 0, device_id: 0, numa_id: 6, shard_devices: [0, 1], cpus: [144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167]
2025-01-07 09:31:33,413 [INFO] [pid: 1607] cpu_binding.py-231: process 1607, new_affinity is [144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155], cpu count 12
2025-01-07 09:31:33,417 [INFO] [pid: 1608] cpu_binding.py-206: rank_id: 1, device_id: 1, numa_id: 6, shard_devices: [0, 1], cpus: [144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167]
2025-01-07 09:31:33,418 [INFO] [pid: 1608] cpu_binding.py-231: process 1608, new_affinity is [156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167], cpu count 12
2025-01-07 09:31:33,535 [INFO] [pid: 1609] cpu_binding.py-206: rank_id: 2, device_id: 2, numa_id: 4, shard_devices: [2, 3], cpus: [96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119]
2025-01-07 09:31:33,537 [INFO] [pid: 1609] cpu_binding.py-231: process 1609, new_affinity is [96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107], cpu count 12
2025-01-07 09:31:33,555 [INFO] [pid: 1610] cpu_binding.py-206: rank_id: 3, device_id: 3, numa_id: 4, shard_devices: [2, 3], cpus: [96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119]
2025-01-07 09:31:33,557 [INFO] [pid: 1610] cpu_binding.py-231: process 1610, new_affinity is [108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119], cpu count 12
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2025-01-07 09:31:34,057 [INFO] [pid: 1607] logging.py-53: model_runner.quantize: None
, model_runner.kv_quant: None
, model_runner.dytpe: torch.float16
2025-01-07 09:31:34,057 [INFO] [pid: 1607] logging.py-53: Rank table file location: 
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())
[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())
[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())
[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())
2025-01-07 09:31:45,555 [INFO] [pid: 1608] dist.py-94: initialize_distributed has been Set
2025-01-07 09:31:46,179 [INFO] [pid: 1608] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.
2025-01-07 09:31:49,000 [INFO] [pid: 1607] dist.py-94: initialize_distributed has been Set
2025-01-07 09:31:49,002 [INFO] [pid: 1607] logging.py-53: init tokenizer done: Qwen2TokenizerFast(name_or_path='/model/Qwen2.5-32B-Instruct', vocab_size=151643, model_max_length=131072, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>', '<|object_ref_start|>', '<|object_ref_end|>', '<|box_start|>', '<|box_end|>', '<|quad_start|>', '<|quad_end|>', '<|vision_start|>', '<|vision_end|>', '<|vision_pad|>', '<|image_pad|>', '<|video_pad|>']}, clean_up_tokenization_spaces=False),  added_tokens_decoder={
        151643: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151644: AddedToken("<|im_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151645: AddedToken("<|im_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151646: AddedToken("<|object_ref_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151647: AddedToken("<|object_ref_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151648: AddedToken("<|box_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151649: AddedToken("<|box_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151650: AddedToken("<|quad_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151651: AddedToken("<|quad_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151652: AddedToken("<|vision_start|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151653: AddedToken("<|vision_end|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151654: AddedToken("<|vision_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151655: AddedToken("<|image_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151656: AddedToken("<|video_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),
        151657: AddedToken("<tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
        151658: AddedToken("</tool_call>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
        151659: AddedToken("<|fim_prefix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
        151660: AddedToken("<|fim_middle|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
        151661: AddedToken("<|fim_suffix|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
        151662: AddedToken("<|fim_pad|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
        151663: AddedToken("<|repo_name|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
        151664: AddedToken("<|file_sep|>", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),
}
2025-01-07 09:31:49,402 [INFO] [pid: 1607] logging.py-53: NPUSocInfo(soc_name='', soc_version=224, need_nz=False)
2025-01-07 09:31:49,549 [INFO] [pid: 1607] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.
2025-01-07 09:31:50,287 [INFO] [pid: 1609] dist.py-94: initialize_distributed has been Set
2025-01-07 09:31:50,731 [INFO] [pid: 1609] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.
2025-01-07 09:31:55,519 [INFO] [pid: 1610] dist.py-94: initialize_distributed has been Set
2025-01-07 09:31:56,131 [INFO] [pid: 1610] flash_causal_qwen2.py-52: >>>> qwen_DecoderModel is called.
Fatal Python error: PyThreadState_Get: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: finalizing (tstate=0x0000ffff8c01e310)

2025-01-07 09:32:02,860 [INFO] [pid: 1607] logging.py-53: model:
 FlashQwen2ForCausalLM(
  (rotary_embedding): PositionRotaryEmbedding()
  (attn_mask): AttentionMask()
  (transformer): FlashQwenModel(
    (wte): TensorParallelEmbedding()
    (h): ModuleList(
      (0-63): 64 x FlashQwenLayer(
        (attn): FlashQwenAttention(
          (rotary_emb): PositionRotaryEmbedding()
          (c_attn): TensorParallelColumnLinear(
            (linear): FastLinear()
          )
          (c_proj): TensorParallelRowLinear(
            (linear): FastLinear()
          )
        )
        (mlp): QwenMLP(
          (act): SiLU()
          (w2_w1): TensorParallelColumnLinear(
            (linear): FastLinear()
          )
          (c_proj): TensorParallelRowLinear(
            (linear): FastLinear()
          )
        )
        (ln_1): QwenRMSNorm()
        (ln_2): QwenRMSNorm()
      )
    )
    (ln_f): QwenRMSNorm()
  )
  (lm_head): TensorParallelHead(
    (linear): FastLinear()
  )
)
Daemon wait pid with 1608, status 134Daemon is killing...
run.sh: line 15:  1591 Killed                  ./bin/mindieservice_daemon