SenseVoice 在npu设备上进行多线程推理偶发假死
收藏回复举报
SenseVoice 在npu设备上进行多线程推理偶发假死
t('forum.solved') 已解决
新人帖
发表于2025-07-15 11:34:06
0 查看

环境

操作系统:openEuler release 24.03 (LTS)

npu:910B4

python:3.10

torch:2.1.0

torch-npu:2.1.0.post8

npu-smi info:

true

SenseVoice模型:参照以下地址把模型改成了om模型,推理部分代码也一并调整。

参考地址:https://gitee.com/ascend/ModelZoo-PyTorch/blob/master/ACL_PyTorch/built-in/audio/SenseVoice/README_onnx.md

python框架:uvicorn+fastapi

其他信息补充

  • 在启动时根据自定义的配置文件初始化n个SenseVoice的mo模型实例到自定义线程池中给语音转文本推理使用,目前n为5
  • 多卡环境应用中使用的模型会加载到指定一张卡

功能场景:通过api传入音频文件,经过vad模型获取音频活动端点,再根据音频活动端点从线程池获取SenseVoice的mo模型实例进行音频转文本推理

问题求助

  • 在运行一段时间之后遇到进行语音转文本推理时假死的情况如下图
  • 假死之后就无法访问任何api了,后续也无任何日志输出
  • 在~/ascend/log/debug/plog 下也没有发现错误日志(之前遇到其他acl的报错能在plog中发现错误日志)

true

SenseVoice推理部分代码

import torch
from transformers import is_torch_npu_available

import time
import numpy as np
from app.log import logger
from funasr import AutoModel
from funasr.utils.load_utils import load_audio_text_image_video, extract_fbank
from app import config
from tqdm import tqdm
from app.util import utils

if utils.is_not_blank(config.device) and "npu" in config.device.lower() and is_torch_npu_available():
    import torch_npu
    from ais_bench.infer.interface import InferSession

class SenseVoiceOnnxModel():
    def __init__(self):
        super().__init__()
        self.blank_id = 0
        self.lid_dict = {"auto": 0, "zh": 3, "en": 4, "yue": 7, "ja": 11, "ko": 12, "nospeech": 13}
        self.textnorm_dict = {'withitn': 14, "woitn": 15}
        self.current_device_index = config.current_device_index
        torch_npu.npu.set_device(self.current_device_index)
        _, kwargs = AutoModel.build_model(model=config.model.auto_speech_recognition.sense_voice.path,
                                          trust_remote_code=False)
        self.om_path = config.model.auto_speech_recognition.sense_voice.om_path
        self.om_sess = InferSession(self.current_device_index, self.om_path)
        self.kwargs = kwargs
        self.transform = True

    def __call__(
            self,
            data_in,
            language=None,
            use_itn=None,
            tokenizer=None,
            frontend=None,
            show_progress=True,
    ):
        results = []
        try:
            torch_npu.npu.set_device(self.current_device_index)
            key = ["wav_file_tmp_name"]
            if use_itn == None:
                use_itn = self.kwargs.get('use_itn', False)
            kwargs = self.kwargs
            # 直接传音频波形的时候transform=True
            if self.transform:
                audio_sample_list = data_in
            else:
                audio_sample_list = load_audio_text_image_video(
                    data_in,
                    fs=frontend.fs,
                    audio_fs=kwargs.get("fs", 16000),
                    data_type=kwargs.get("data_type", "sound"),
                    tokenizer=tokenizer,
                )

            data_type = kwargs.get("data_type", "sound")

            if frontend == None:
                frontend = kwargs.get('frontend')
            frontend_conf = kwargs.get('frontend_conf')
            if tokenizer == None:
                tokenizer = kwargs.get('tokenizer')
            tokenizer_conf = kwargs.get('tokenizer_conf')
            if show_progress:
                progress_bar = tqdm(
                    total=1,
                    desc="segment for asr processing",
                    unit="frames",
                    bar_format="{l_bar}{bar}{r_bar} | {percentage:.2f}%",
                )
            speech, speech_lengths = extract_fbank(
                audio_sample_list, data_type=kwargs.get("data_type", "sound"), frontend=frontend
            )
            speech = speech.to(device=kwargs["device"])
            speech_lengths = speech_lengths.to(device=kwargs["device"])

            if language == None:
                language = kwargs.get("language", "auto")
            language = torch.LongTensor([self.lid_dict[language] if language in self.lid_dict else 0]).to(speech.device)

            textnorm = kwargs.get("text_norm", None)
            if textnorm is None:
                textnorm = "withitn" if use_itn else "woitn"
            textnorm = torch.LongTensor([self.textnorm_dict[textnorm]]).to(speech.device)

            s = time.time()
            feed = [speech.cpu().detach().numpy().astype(np.float32),
                    speech_lengths.cpu().detach().numpy().astype(np.int32),
                    language.cpu().detach().numpy().astype(np.int32),
                    textnorm.cpu().detach().numpy().astype(np.int32)]

            ctc_logits, encoder_out_lens = self.om_sess.infer(feed, mode='dymshape', custom_sizes=300000000)
            ctc_logits = torch.from_numpy(ctc_logits).npu()
            encoder_out_lens = torch.from_numpy(encoder_out_lens).npu()
            e = time.time()
            cost_time = e - s



            x = ctc_logits[0, : encoder_out_lens[0].item(), :]
            yseq = x.argmax(dim=-1)
            yseq = torch.unique_consecutive(yseq, dim=-1)
            mask = yseq != self.blank_id
            token_int = yseq[mask].tolist()

            # Change integer-ids to tokens
            text = tokenizer.decode(token_int)
            result_i = {"key": key[0], "text": text}
            results.append(result_i)
            if show_progress:
                progress_bar.update(1)
        except Exception as e:
            logger.error(f"Sensevoice Inference failed: {e}")
        return results

本帖最后由 匿名用户2025/07/15 12:09:06 编辑

我要发帖子