参考https://gitee.com/ascend/ModelZoo-PyTorch/blob/master/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/README.md
固件版本:
Version=7.7.0.1.231
firmware_version=1.0
package_version=25.0.rc1.1
compatible_version_drv=[23.0.rc2,23.0.rc2.],[23.0.rc3,23.0.rc3.],[23.0.0,23.0.0.],[24.0,24.0.],[24.1,24.1.],[25.0,25.0.]
驱动版本:
Version=25.0.rc1.1
ascendhal_version=7.35.23
aicpu_version=1.0
tdt_version=1.0
log_version=1.0
prof_version=2.0
dvppkernels_version=1.1
tsfw_version=1.0
Innerversion=V100R001C21SPC002B220
compatible_version=[V100R001C17],[V100R001C18],[V100R001C19],[V100R001C20],[V100R001C21]
compatible_version_fw=[6.4.0,6.4.99],[7.0.0,7.7.99]
package_version=25.0.rc1.1
docker环境与文档中完全一致
在300I DUO卡上进行CosyVoice2的推理部署测试，在模型分割、模型转换中均无问题，但在执行最后的推理时发生报错：
RuntimeError: EZ1001: [PID: 235107] 2025-06-29-17:22:50.399.358 PromptFlashAttention LaunchAicore failed.
TraceBack (most recent call last):
attention mask must be NULL, when Qs,Kvs is unAlign or Qs is not equal to Kvs, Qs = 53, Kvs = 53[FUNC:RunBigKernelTilingWithParams][FILE:prompt_flash_attention_tiling.cpp][LINE:3555]
Tiling failed
Tiling Failed.
Kernel GetWorkspace failed. opType: 18
PromptFlashAttention LaunchAicore failed.
call aclnnPromptFlashAttentionV3 failed:[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:119]
api_ret faild:4294967295[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:122]
Assert ((op_execute_func(op_execute_context)) == ge::SUCCESS) failed[FUNC:ExecuteOpFunc][FILE:aclnn_op_execute_kernel.cc][LINE:47]
Failed to execute rt v2 model for graph graph_1_0, model_id 4.[FUNC:ExecuteWithStreamAsync][FILE:hybrid_model_rt_v2_executor.cc][LINE:1069]
GraphManager ExecuteGrapWithStreamhAsync failed,session id = 1, graph id = 0, stream = 0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:inner_session.cc][LINE:577]
Execute graph with stream async failed, error code:1343225857, session_id:1, graph_id:0, stream:0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:ge_api.cc][LINE:883]
Traceback (most recent call last):
File "/usr/local/lib64/python3.11/site-packages/torch_npu/dynamo/torchair/_utils/error_code.py", line 43, in wapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch_npu/dynamo/torchair/core/_backend.py", line 133, in run
return super(TorchNpuGraph, self).run((inputs, assigned_outputs, stream))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: EZ1001: [PID: 235107] 2025-06-29-17:22:50.399.358 PromptFlashAttention LaunchAicore failed.
TraceBack (most recent call last):
attention mask must be NULL, when Qs,Kvs is unAlign or Qs is not equal to Kvs, Qs = 53, Kvs = 53[FUNC:RunBigKernelTilingWithParams][FILE:prompt_flash_attention_tiling.cpp][LINE:3555]
Tiling failed
Tiling Failed.
Kernel GetWorkspace failed. opType: 18
PromptFlashAttention LaunchAicore failed.
call aclnnPromptFlashAttentionV3 failed:[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:119]
api_ret faild:4294967295[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:122]
Assert ((op_execute_func(op_execute_context)) == ge::SUCCESS) failed[FUNC:ExecuteOpFunc][FILE:aclnn_op_execute_kernel.cc][LINE:47]
Failed to execute rt v2 model for graph graph_1_0, model_id 4.[FUNC:ExecuteWithStreamAsync][FILE:hybrid_model_rt_v2_executor.cc][LINE:1069]
GraphManager ExecuteGrapWithStreamhAsync failed,session id = 1, graph id = 0, stream = 0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:inner_session.cc][LINE:577]
Execute graph with stream async failed, error code:1343225857, session_id:1, graph_id:0, stream:0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:ge_api.cc][LINE:883]
[ERROR] 2025-06-29-17:22:50 (PID:235107, Device:0, RankID:-1) ERR03005 GRAPH internal error
0%| | 0/1 [13:34<?, ?it/s]
Traceback (most recent call last):
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/infer.py", line 103, in <module>
no_stream_input_inference(args, cosyvoice, prompt_txt)
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/infer.py", line 27, in no_stream_input_inference
for _ in enumerate(cosyvoice.inference_sft(prompt_txt[0], '中文女', stream=args.stream_out)):
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/cli/cosyvoice.py", line 74, in inference_sft
for model_output in self.model.tts(**model_input, stream=stream, speed=speed):
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/cli/model.py", line 414, in tts
this_tts_speech = self.token2wav(token=this_tts_speech_token,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/cli/model.py", line 334, in token2wav
tts_mel, _ = self.flow.inference(token=token.to(self.device),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/flow/flow.py", line 215, in inference
token = self.input_embedding(torch.clamp(token, min=0)) * mask
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/sparse.py", line 164, in forward
return F.embedding(
^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/functional.py", line 2267, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: call aclnnEmbedding failed, detail:EZ1001: [PID: 235107] 2025-06-29-17:22:50.455.223 indices not implemented for DT_FLOAT, should be in dtype support list [DT_INT32,DT_INT64,].
是代码问题还是算子适配问题，暂不支持300I DUO卡？我在文档里只看到了800I A2的性能数据
点赞
收藏
回复
分享
举报
已解决
参考https://gitee.com/ascend/ModelZoo-PyTorch/blob/master/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/README.md
固件版本:
Version=7.7.0.1.231
firmware_version=1.0
package_version=25.0.rc1.1
compatible_version_drv=[23.0.rc2,23.0.rc2.],[23.0.rc3,23.0.rc3.],[23.0.0,23.0.0.],[24.0,24.0.],[24.1,24.1.],[25.0,25.0.]
驱动版本:
Version=25.0.rc1.1
ascendhal_version=7.35.23
aicpu_version=1.0
tdt_version=1.0
log_version=1.0
prof_version=2.0
dvppkernels_version=1.1
tsfw_version=1.0
Innerversion=V100R001C21SPC002B220
compatible_version=[V100R001C17],[V100R001C18],[V100R001C19],[V100R001C20],[V100R001C21]
compatible_version_fw=[6.4.0,6.4.99],[7.0.0,7.7.99]
package_version=25.0.rc1.1
docker环境与文档中完全一致
在300I DUO卡上进行CosyVoice2的推理部署测试，在模型分割、模型转换中均无问题，但在执行最后的推理时发生报错：
RuntimeError: EZ1001: [PID: 235107] 2025-06-29-17:22:50.399.358 PromptFlashAttention LaunchAicore failed.
TraceBack (most recent call last):
attention mask must be NULL, when Qs,Kvs is unAlign or Qs is not equal to Kvs, Qs = 53, Kvs = 53[FUNC:RunBigKernelTilingWithParams][FILE:prompt_flash_attention_tiling.cpp][LINE:3555]
Tiling failed
Tiling Failed.
Kernel GetWorkspace failed. opType: 18
PromptFlashAttention LaunchAicore failed.
call aclnnPromptFlashAttentionV3 failed:[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:119]
api_ret faild:4294967295[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:122]
Assert ((op_execute_func(op_execute_context)) == ge::SUCCESS) failed[FUNC:ExecuteOpFunc][FILE:aclnn_op_execute_kernel.cc][LINE:47]
Failed to execute rt v2 model for graph graph_1_0, model_id 4.[FUNC:ExecuteWithStreamAsync][FILE:hybrid_model_rt_v2_executor.cc][LINE:1069]
GraphManager ExecuteGrapWithStreamhAsync failed,session id = 1, graph id = 0, stream = 0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:inner_session.cc][LINE:577]
Execute graph with stream async failed, error code:1343225857, session_id:1, graph_id:0, stream:0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:ge_api.cc][LINE:883]
Traceback (most recent call last):
File "/usr/local/lib64/python3.11/site-packages/torch_npu/dynamo/torchair/_utils/error_code.py", line 43, in wapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch_npu/dynamo/torchair/core/_backend.py", line 133, in run
return super(TorchNpuGraph, self).run((inputs, assigned_outputs, stream))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: EZ1001: [PID: 235107] 2025-06-29-17:22:50.399.358 PromptFlashAttention LaunchAicore failed.
TraceBack (most recent call last):
attention mask must be NULL, when Qs,Kvs is unAlign or Qs is not equal to Kvs, Qs = 53, Kvs = 53[FUNC:RunBigKernelTilingWithParams][FILE:prompt_flash_attention_tiling.cpp][LINE:3555]
Tiling failed
Tiling Failed.
Kernel GetWorkspace failed. opType: 18
PromptFlashAttention LaunchAicore failed.
call aclnnPromptFlashAttentionV3 failed:[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:119]
api_ret faild:4294967295[FUNC:PromptHostExecuteFunc][FILE:fallback_prompt_flash_attention.cpp][LINE:122]
Assert ((op_execute_func(op_execute_context)) == ge::SUCCESS) failed[FUNC:ExecuteOpFunc][FILE:aclnn_op_execute_kernel.cc][LINE:47]
Failed to execute rt v2 model for graph graph_1_0, model_id 4.[FUNC:ExecuteWithStreamAsync][FILE:hybrid_model_rt_v2_executor.cc][LINE:1069]
GraphManager ExecuteGrapWithStreamhAsync failed,session id = 1, graph id = 0, stream = 0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:inner_session.cc][LINE:577]
Execute graph with stream async failed, error code:1343225857, session_id:1, graph_id:0, stream:0xaaadfc088e40.[FUNC:ExecuteGraphWithStreamAsync][FILE:ge_api.cc][LINE:883]
[ERROR] 2025-06-29-17:22:50 (PID:235107, Device:0, RankID:-1) ERR03005 GRAPH internal error
0%| | 0/1 [13:34<?, ?it/s]
Traceback (most recent call last):
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/infer.py", line 103, in <module>
no_stream_input_inference(args, cosyvoice, prompt_txt)
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/infer.py", line 27, in no_stream_input_inference
for _ in enumerate(cosyvoice.inference_sft(prompt_txt[0], '中文女', stream=args.stream_out)):
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/cli/cosyvoice.py", line 74, in inference_sft
for model_output in self.model.tts(**model_input, stream=stream, speed=speed):
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/cli/model.py", line 414, in tts
this_tts_speech = self.token2wav(token=this_tts_speech_token,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/cli/model.py", line 334, in token2wav
tts_mel, _ = self.flow.inference(token=token.to(self.device),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/openlab/ljh-cosyvoice2/CosyVoice2-05b/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice2/CosyVoice/cosyvoice/flow/flow.py", line 215, in inference
token = self.input_embedding(torch.clamp(token, min=0)) * mask
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/sparse.py", line 164, in forward
return F.embedding(
^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/functional.py", line 2267, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: call aclnnEmbedding failed, detail:EZ1001: [PID: 235107] 2025-06-29-17:22:50.455.223 indices not implemented for DT_FLOAT, should be in dtype support list [DT_INT32,DT_INT64,].
是代码问题还是算子适配问题，暂不支持300I DUO卡？我在文档里只看到了800I A2的性能数据