(cosyvoice2) [root@localhost CosyVoice]# python3 infer.py --model_path=/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice-300M --campplus=/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/campplus_linux_aarch64.om --speech=//home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/speech_linux_aarch64.om --flow=/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/flow_linux_aarch64.om
/usr/local/lib64/python3.9/site-packages/torch_npu/contrib/transfer_to_npu.py:305: ImportWarning:
    *************************************************************************************************************
    The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
    The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
    The backend in torch.distributed.init_process_group set to hccl now..
    The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
    The device parameters have been replaced with npu in the function below:
    torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.autocast, torch.load, torch.Generator, torch.set_default_device, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.nn.Module.to, torch.nn.Module.to_empty
    *************************************************************************************************************

  warnings.warn(msg, ImportWarning)
/usr/local/lib64/python3.9/site-packages/torch_npu/contrib/transfer_to_npu.py:260: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, which currently does not support them, if you need to enable them, please do not use transfer_to_npu.
  warnings.warn(msg, RuntimeWarning)
/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/__init__.py:3: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py:3142: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
2025-02-10 18:44:39,091 - modelscope - INFO - PyTorch version 2.4.0 Found.
2025-02-10 18:44:39,094 - modelscope - INFO - Loading ast index from /root/.cache/modelscope/ast_indexer
2025-02-10 18:44:39,094 - modelscope - INFO - No valid ast index found from /root/.cache/modelscope/ast_indexer, generating ast index from prebuilt!
2025-02-10 18:44:39,198 - modelscope - INFO - Loading done! Current index file version is 1.15.0, with md5 5c903095c7fc289290d42bb125915c58 and a total number of 980 components indexed
transformer is not installed, please install it if you want to use related modules
failed to import ttsfrd, use WeTextProcessing instead
/usr/local/lib64/python3.9/site-packages/torch/_jit_internal.py:745: FutureWarning: ignore(True) has been deprecated. TorchScript will now drop the function call on compilation. Use torch.jit.unused now. {}
  warnings.warn(
/usr/local/lib/python3.9/site-packages/lightning/fabric/__init__.py:41: Deprecated call to `pkg_resources.declare_namespace('lightning.fabric')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py:2554: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('lightning')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(parent)
[INFO] Failed to extract font properties from /usr/share/fonts/cantarell/Cantarell-VF.otf: In FT2Font: Can not load face (SFNT font table missing; error code 0x8e)
[INFO] generated new fontManager
/usr/local/lib/python3.9/site-packages/lightning/pytorch/__init__.py:37: Deprecated call to `pkg_resources.declare_namespace('lightning.pytorch')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py:2554: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('lightning')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(parent)
/usr/local/lib/python3.9/site-packages/diffusers/models/lora.py:393: FutureWarning: `LoRACompatibleLinear` is deprecated and will be removed in version 1.0.0. Use of `LoRACompatibleLinear` is deprecated. Please switch to PEFT backendby installing PEFT: `pip install peft`.
  deprecate("LoRACompatibleLinear", "1.0.0", deprecation_message)
[INFO] input frame rate=50
/usr/local/lib64/python3.9/site-packages/torch/nn/utils/weight_norm.py:134: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.
  WeightNorm.apply(module, name, dim)
/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/dataset/processor.py:24: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You canremove the function call.
  torchaudio.set_audio_backend('soundfile')
/usr/local/lib/python3.9/site-packages/whisper/tokenizer.py:333: ResourceWarning: unclosed file <_io.TextIOWrapper name='/usr/local/lib/python3.9/site-packages/whisper/assets/multilingual.tiktoken' mode='r' encoding='UTF-8'>
  ranks = {
/usr/local/lib64/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
2025-02-10 18:44:59,616 WETEXT INFO found existing fst: /usr/local/lib/python3.9/site-packages/tn/zh_tn_tagger.fst
[INFO] found existing fst: /usr/local/lib/python3.9/site-packages/tn/zh_tn_tagger.fst
2025-02-10 18:44:59,616 WETEXT INFO                     /usr/local/lib/python3.9/site-packages/tn/zh_tn_verbalizer.fst
[INFO]                     /usr/local/lib/python3.9/site-packages/tn/zh_tn_verbalizer.fst
2025-02-10 18:44:59,616 WETEXT INFO skip building fst for zh_normalizer ...
[INFO] skip building fst for zh_normalizer ...
2025-02-10 18:45:00,269 WETEXT INFO found existing fst: /usr/local/lib/python3.9/site-packages/tn/en_tn_tagger.fst
[INFO] found existing fst: /usr/local/lib/python3.9/site-packages/tn/en_tn_tagger.fst
2025-02-10 18:45:00,269 WETEXT INFO                     /usr/local/lib/python3.9/site-packages/tn/en_tn_verbalizer.fst
[INFO]                     /usr/local/lib/python3.9/site-packages/tn/en_tn_verbalizer.fst
2025-02-10 18:45:00,269 WETEXT INFO skip building fst for en_normalizer ...
[INFO] skip building fst for en_normalizer ...
[WARN] acl repeat initialize
[INFO] acl init success
[INFO] open device 0 success
[INFO] get current context
[INFO] load model /home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/flow_linux_aarch64.om success
[INFO] create model description success
[INFO] create new context
[INFO] load model //home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/speech_linux_aarch64.om success
[INFO] create model description success
[INFO] create new context
[INFO] load model /home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/campplus_linux_aarch64.om success
[INFO] create model description success
warm up start
Traceback (most recent call last):
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/infer.py", line 69, in <module>
    out = cosyvoice.inference_zero_shot(prompt_txt, out_txt, prompt_speech_16k, flow, speech, campplus)
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/cli/cosyvoice.py", line 60, in inference_zero_shot
    model_output = self.model.inference(o1, **model_input)
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/cli/model.py", line 40, in inference
    tts_speech_token = self.llm.inference(text=text.to(self.device),
  File "/usr/local/lib64/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/llm/llm.py", line 196, in inference
    y_pred, att_cache = self.llm(lm_input, offset=0, required_cache_size=-1, att_cache=att_cache, cnn_cache=cnn_cache,
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/eval_frame.py", line 433, in _fn
    return fn(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 1116, in __call__
    return self._torchdynamo_orig_callable(
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 472, in __call__
    return _compile(
  File "/usr/local/lib64/python3.9/site-packages/torch/_utils_internal.py", line 84, in wrapper_function
    return StrobelightCompileTimeProfiler.profile_compile_time(
  File "/usr/local/lib64/python3.9/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time
    return func(*args, **kwargs)
  File "/usr/lib64/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/bytecode_transformation.py", line 1185, in transform_code_object
    transformations(instructions, code_options)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn
    return fn(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/convert_frame.py", line 582, in transform
    tracer.run()
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2451, in run
    super().run()
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 893, in run
    while self.step():
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 805, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2642, in RETURN_VALUE
    self._return(inst)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/symbolic_convert.py", line 2627, in _return
    self.output.compile_subgraph(
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/output_graph.py", line 1123, in compile_subgraph
    self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
  File "/usr/lib64/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/output_graph.py", line 1318, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/output_graph.py", line 1409, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/output_graph.py", line 1390, in call_user_compiler
    compiled_fn = compiler_fn(gm, self.example_inputs())
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/repro/after_dynamo.py", line 129, in __call__
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/repro/after_dynamo.py", line 129, in __call__
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/usr/local/lib64/python3.9/site-packages/torch/__init__.py", line 1990, in __call__
    return self.compiler_fn(model_, inputs_, **self.kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 460, in _npu_backend
    return aot_module_simplified(gm, example_inputs, fw_compiler=fw_compiler, bw_compiler=compiler,
  File "/usr/local/lib64/python3.9/site-packages/torch/_functorch/aot_autograd.py", line 954, in aot_module_simplified
    compiled_fn, _ = create_aot_dispatcher_function(
  File "/usr/local/lib64/python3.9/site-packages/torch/_dynamo/utils.py", line 231, in time_wrapper
    r = func(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/_functorch/aot_autograd.py", line 687, in create_aot_dispatcher_function
    compiled_fn, fw_metadata = compiler_fn(
  File "/usr/local/lib64/python3.9/site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 168, in aot_dispatch_base
    compiled_fw = compiler(fw_module, updated_flat_args)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 433, in gear_compiler
    return compiler(gm, example_inputs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 408, in wrapped_compiler
    return compiler(gm, example_inputs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/_utils/error_code.py", line 46, in wapper
    raise type(e)("\n".join(msg))
torch._dynamo.exc.BackendCompilerFailed: backend="functools.partial(<function _npu_backend at 0xfffdbe469280>, compiler_config=<torchair.configs.compiler_config.CompilerConfig object at 0xfffe763da370>, decompositions={<OpOverload(op='npu_define.all_to_all_single', overload='default')>: <function all_to_all_single_decomposition at 0xfffdbdd51f70>, <OpOverload(op='npu_define.all_to_all', overload='default')>: <function all_to_all_decomposition at 0xfffdbdd5b4c0>, <OpOverload(op='npu_define.allgather', overload='default')>: <function allgather_decomposition at 0xfffdbdd51d30>})" raised:
RuntimeError: Unsupported torch op aten.sym_size.int by ge

While executing %sym_size_int_2 : [num_users=126] = call_function[target=torch.ops.aten.sym_size.int](args = (%slice_4, 1), kwargs = {})
Original traceback:
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/transformer/encoder.py", line 419, in forward
    xs, _, new_att_cache, new_cnn_cache = layer(
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/transformer/encoder_layer.py", line 93, in forward
    x_att, new_att_cache = self.self_attn(x, x, x, mask, pos_emb=pos_emb, cache=att_cache)
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/transformer/attention.py", line 301, in forward
    p = self.linear_pos(pos_emb).view(n_batch_pos, -1, self.h, self.d_k)

Traceback (most recent call last):
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/_utils/error_code.py", line 43, in wapper
    return func(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 320, in __call__
    return self._get_compiled_gm(gm, example_inputs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 356, in _get_compiled_gm
    return _GmRunner(self._gen_compiled_gm(gm, example_inputs))
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 366, in _gen_compiled_gm
    concrete_graph: ConcreteGraphBase = _NpuGraphConverter(
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 151, in run
    super().run(*args, **kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch/fx/interpreter.py", line 146, in run
    self.env[node] = self.run_node(node)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 145, in run_node
    return super().run_node(n)
  File "/usr/local/lib64/python3.9/site-packages/torch/fx/interpreter.py", line 203, in run_node
    return getattr(self, n.op)(n.target, args, kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 118, in inner
    result = f(self, target, args, kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 205, in call_function
    return self._wrap('call_function')(target, args, kwargs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 192, in inner
    npu_outputs = self._graph.parse_node(target, args_npu, kwargs_npu, meta_outputs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/_ge_concrete_graph/continguous_utils.py", line 149, in wrapper
    return func(self, target, args_new, kwargs_new, meta_outputs)
  File "/usr/local/lib64/python3.9/site-packages/torch_npu/dynamo/torchair/_ge_concrete_graph/fx2ge_converter.py", line 826, in parse_node
    raise RuntimeError(f"Unsupported torch op {target} by ge")
RuntimeError: Unsupported torch op aten.sym_size.int by ge

While executing %sym_size_int_2 : [num_users=126] = call_function[target=torch.ops.aten.sym_size.int](args = (%slice_4, 1), kwargs = {})
Original traceback:
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/transformer/encoder.py", line 419, in forward
    xs, _, new_att_cache, new_cnn_cache = layer(
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/transformer/encoder_layer.py", line 93, in forward
    x_att, new_att_cache = self.self_attn(x, x, x, mask, pos_emb=pos_emb, cache=att_cache)
  File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ModelZoo-PyTorch/ACL_PyTorch/built-in/audio/CosyVoice/CosyVoice/cosyvoice/transformer/attention.py", line 301, in forward
    p = self.linear_pos(pos_emb).view(n_batch_pos, -1, self.h, self.d_k)



[ERROR] 2025-02-10-18:47:46 (PID:2564357, Device:0, RankID:-1) ERR03005 GRAPH internal error

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

[INFO] unload model success, model Id is 2147483648
[INFO] unload model success, model Id is 2147483649
[INFO] unload model success, model Id is 2147483650