今日论文合集：cs.SD语音8篇，eess.AS音频处理9篇。
cs.SD语音
【1】 Backdoor Attacks against Voice Recognition Systems: A Survey标题：针对语音识别系统的后门攻击：综述链接：https://arxiv.org/abs/2307.13643
作者：Baochen Yan,Jiahe Lan,Zheng Yan机构：Xidian University备注：33 pages, 7 figures摘要：语音识别系统（VRS）采用深度学习进行语音识别和说话人识别。它们已被广泛部署在各种现实应用中，从智能语音辅助到电话监控和生物识别认证。然而，先前的研究已经揭示了VRS的脆弱性后门攻击，这对VRS的安全性和隐私性构成了重大威胁。不幸的是，现有的文献缺乏一个彻底的审查这个问题。本文填补了这一研究空白，进行了全面的调查后门攻击VRS。我们首先介绍了VRS和后门攻击的概述，阐明了它们的基本知识。然后，我们提出了一套评估标准来评估后门攻击方法的性能。其次，从不同角度对虚拟现实后门攻击进行了分类，并分析了不同类型的后门攻击的特点。在此之后，我们全面审查现有的攻击方法，并分析其优点和缺点的基础上提出的标准。此外，我们回顾了经典的后门防御方法和通用的音频防御技术。然后，我们讨论了部署在虚拟现实系统的可行性。最后，我们指出了几个开放的问题，并进一步提出了未来的研究方向，以推动研究的虚拟现实系统的安全性。摘要：Voice Recognition Systems (VRSs) employ deep learning for speech recognition and speaker recognition. They have been widely deployed in various real-world applications, from intelligent voice assistance to telephony surveillance and biometric authentication. However, prior research has revealed the vulnerability of VRSs to backdoor attacks, which pose a significant threat to the security and privacy of VRSs. Unfortunately, existing literature lacks a thorough review on this topic. This paper fills this research gap by conducting a comprehensive survey on backdoor attacks against VRSs. We first present an overview of VRSs and backdoor attacks, elucidating their basic knowledge. Then we propose a set of evaluation criteria to assess the performance of backdoor attack methods. Next, we present a comprehensive taxonomy of backdoor attacks against VRSs from different perspectives and analyze the characteristic of different categories. After that, we comprehensively review existing attack methods and analyze their pros and cons based on the proposed criteria. Furthermore, we review classic backdoor defense methods and generic audio defense techniques. Then we discuss the feasibility of deploying them on VRSs. Finally, we figure out several open issues and further suggest future research directions to motivate the research of VRSs security.
【2】 Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals  using Self Supervised Speech Representations标题：使用自我监督语音表征的听障人士非侵入性可理解性预报器链接：https://arxiv.org/abs/2307.13423
作者：George Close,Thomas Hain,Stefan Goetze机构：Speech and Hearing Group, Department of Computer Science, University of Sheffield, United Kingdom备注：Submitted to ASRU 2023摘要：自监督语音表示（SSSR）已经成功地应用于许多语音处理任务，例如：作为用于语音质量（SQ）预测的特征提取器，其进而与用于具有正常或受损听力的用户的评估和训练语音增强系统相关。然而，确切的知识，为什么以及如何编码质量相关的信息，以及在这样的表示仍然知之甚少。在这项工作中，技术的非侵入性预测的SQ评级扩展到预测的听力受损的用户的可懂度。据发现，自我监督的表示是有用的输入功能，非侵入性的预测模型，实现更复杂的系统的竞争力的表现。对清晰度预测挑战1听众和增强系统性能的详细分析表明，可能需要更多的数据才能推广到未知系统和（听力受损）个人摘要：Self-supervised speech representations (SSSRs) have been successfully applied to a number of speech-processing tasks, e.g. as feature extractor for speech quality (SQ) prediction, which is, in turn, relevant for assessment and training speech enhancement systems for users with normal or impaired hearing. However, exact knowledge of why and how quality-related information is encoded well in such representations remains poorly understood. In this work, techniques for non-intrusive prediction of SQ ratings are extended to the prediction of intelligibility for hearing-impaired users. It is found that self-supervised representations are useful as input features to non-intrusive prediction models, achieving competitive performance to more complex systems. A detailed analysis of the performance depending on Clarity Prediction Challenge 1 listeners and enhancement systems indicates that more data might be needed to allow generalisation to unknown systems and (hearing-impaired) individuals
【3】 A Snoring Sound Dataset for Body Position Recognition: Collection,  Annotation, and Analysis标题：用于体位识别的鼾声数据集：采集、标注和分析链接：https://arxiv.org/abs/2307.13346
作者：Li Xiao,Xiuping Yang,Xinhong Li,Weiping Tu,Xiong Chen,Weiyan Yi,Jie Lin,Yuhong Yang,Yanzhen Ren机构：NERCMS, School of Computer Science, Wuhan University, Sleep Medicine Centre, Zhongnan Hospital of Wuhan University, School of Cyber Science and Engineering, Wuhan University备注：Accepted to INTERSPEECH 2023摘要：阻塞性睡眠呼吸暂停低通气综合征（OSAHS）是由上呼吸道阻塞引起的慢性呼吸障碍。打鼾是OSAHS的突出症状，并且先前的研究试图通过打鼾声音来识别上呼吸道的阻塞部位。尽管取得了一些进展，但由于睡眠体位对上呼吸道的影响，阻塞部位的分类在现实世界的临床环境中仍然具有挑战性。为了解决这一挑战，本文提出了一个基于打鼾的睡眠体位识别数据集（SSBPR），该数据集由7570个打鼾记录组成，包括六个不同的睡眠体位标签：仰卧、仰卧但左侧头、仰卧但右侧头、左侧卧、右侧卧和俯卧。实验结果表明，打鼾声音表现出某些声学特征，使得它们能够有效地用于在现实世界场景中的睡眠期间识别身体姿势。摘要：Obstructive Sleep Apnea-Hypopnea Syndrome (OSAHS) is a chronic breathing disorder caused by a blockage in the upper airways. Snoring is a prominent symptom of OSAHS, and previous studies have attempted to identify the obstruction site of the upper airways by snoring sounds. Despite some progress, the classification of the obstruction site remains challenging in real-world clinical settings due to the influence of sleep body position on upper airways. To address this challenge, this paper proposes a snore-based sleep body position recognition dataset (SSBPR) consisting of 7570 snoring recordings, which comprises six distinct labels for sleep body position: supine, supine but left lateral head, supine but right lateral head, left-side lying, right-side lying and prone. Experimental results show that snoring sounds exhibit certain acoustic features that enable their effective utilization for identifying body posture during sleep in real-world scenarios.
【4】 CQNV: A combination of coarsely quantized bitstream and neural vocoder  for low rate speech coding标题：CQNV：用于低速率语音编码的粗量化比特流和神经声码器的组合链接：https://arxiv.org/abs/2307.13295
作者：Youqiang Zheng,Li Xiao,Weiping Tu,Yuhong Yang,Xinmeng Xu机构：NERCMS, School of Computer Science, Wuhan University, China, Hubei Luojia Laboratory, China, Hubei Key Laboratory of Multimedia and Network Communication Engineering备注：Accepted by INTERSPEECH 2023摘要：最近，基于神经网络的语音编解码器已被证明比传统方法表现更好。然而，传统参数量化中的冗余在将传统编解码器与神经声码器组合的编解码器架构内是可见的。在本文中，我们提出了一种新的框架命名为CQNV，它结合了传统的参数化编解码器的粗量化参数，以降低比特率与神经声码器，以提高解码语音的质量。此外，我们引入了一个参数处理模块的神经声码器，以加强传统的语音编码参数的比特流的神经声码器的应用，进一步提高重建语音的质量。在实验中，主观和客观的评价证明了所提出的CQNV框架的有效性。具体来说，我们提出的方法可以实现更高质量的重建语音在1.1 kbps比天琴座和Encodec在3 kbps。摘要：Recently, speech codecs based on neural networks have proven to perform better than traditional methods. However, redundancy in traditional parameter quantization is visible within the codec architecture of combining the traditional codec with the neural vocoder. In this paper, we propose a novel framework named CQNV, which combines the coarsely quantized parameters of a traditional parametric codec to reduce the bitrate with a neural vocoder to improve the quality of the decoded speech. Furthermore, we introduce a parameters processing module into the neural vocoder to enhance the application of the bitstream of traditional speech coding parameters to the neural vocoder, further improving the reconstructed speech's quality. In the experiments, both subjective and objective evaluations demonstrate the effectiveness of the proposed CQNV framework. Specifically, our proposed method can achieve higher quality reconstructed speech at 1.1 kbps than Lyra and Encodec at 3 kbps.
【5】 Audio-aware Query-enhanced Transformer for Audio-Visual Segmentation标题：用于视听分割的音频感知查询增强型转换器链接：https://arxiv.org/abs/2307.13236
作者：Jinxiang Liu,Chen Ju,Chaofan Ma,Yanfeng Wang,Yu Wang,Ya Zhang机构：ShanghaiJiao Tong University备注：arXiv admin note: text overlap with arXiv:2305.11019摘要：视听分割（AVS）任务的目标是使用音频线索来分割视频帧中的发声对象。然而，目前基于融合的方法有性能的限制，由于小的卷积和视听功能的融合不足的接受域。为了克服这些问题，我们提出了一种新的\textbf{Au}dio-aware查询增强的\textbf{TR}ansformer（AuTR）来处理任务。与现有的方法不同，我们的方法引入了一个多模态Transformer架构，使深度融合和聚合的视听功能。此外，我们设计了一个音频感知的查询增强的Transformer解码器，明确地帮助模型专注于基于音频信号的精确定位的发声对象的分割，而忽略沉默但突出的对象。实验结果表明，该方法优于以往的方法，并表现出更好的泛化能力，在多声音和开放集的情况下。摘要：The goal of the audio-visual segmentation (AVS) task is to segment the sounding objects in the video frames using audio cues. However, current fusion-based methods have the performance limitations due to the small receptive field of convolution and inadequate fusion of audio-visual features. To overcome these issues, we propose a novel \textbf{Au}dio-aware query-enhanced \textbf{TR}ansformer (AuTR) to tackle the task. Unlike existing methods, our approach introduces a multimodal transformer architecture that enables deep fusion and aggregation of audio-visual features. Furthermore, we devise an audio-aware query-enhanced transformer decoder that explicitly helps the model focus on the segmentation of the pinpointed sounding objects based on audio signals, while disregarding silent yet salient objects. Experimental results show that our method outperforms previous methods and demonstrates better generalization ability in multi-sound and open-set scenarios.
【6】 Joint speech and overlap detection: a benchmark over multiple audio  setup and speech domains标题：联合语音和重叠检测：多个音频设置和语音域的基准链接：https://arxiv.org/abs/2307.13012
作者：Martin Lebourdais,Théo Mariotte,Marie Tahon,Anthony Larcher,Antoine Laurent,Silvio Montresor,Sylvain Meignier,Jean-Hugh Thomas机构：LIUM,LAUM UMR , IA-GS, Le Mans Universit´e摘要：语音活动和重叠语音检测（分别为VAD和OSD）是说话人日志化的关键预处理任务。最终的分割性能高度依赖于这些子任务的鲁棒性。最近的研究表明，VAD和OSD可以使用多类分类模型联合训练。然而，这些工作往往局限于一个特定的语音域，缺乏有关系统的泛化能力的信息。本文提出了一个完整的和新的基准不同的VAD和OSD模型，在多个音频设置（单/多通道）和语音域（例如，音频）。媒体、会议.）。我们的2/3类系统，结合了时间卷积网络与语音表示适应的设置，优于国家的最先进的结果。我们表明，这两个任务的联合训练提供了类似的性能，在F1分数两个专用的VAD和OSD系统，同时降低培训成本。这种独特的架构也可用于单声道和多声道语音处理。摘要：Voice activity and overlapped speech detection (respectively VAD and OSD) are key pre-processing tasks for speaker diarization. The final segmentation performance highly relies on the robustness of these sub-tasks. Recent studies have shown VAD and OSD can be trained jointly using a multi-class classification model. However, these works are often restricted to a specific speech domain, lacking information about the generalization capacities of the systems. This paper proposes a complete and new benchmark of different VAD and OSD models, on multiple audio setups (single/multi-channel) and speech domains (e.g. media, meeting...). Our 2/3-class systems, which combine a Temporal Convolutional Network with speech representations adapted to the setup, outperform state-of-the-art results. We show that the joint training of these two tasks offers similar performances in terms of F1-score to two dedicated VAD and OSD systems while reducing the training cost. This unique architecture can also be used for single and multichannel speech processing.
【7】 On-Device Speaker Anonymization of Acoustic Embeddings for ASR based  onFlexible Location Gradient Reversal Layer标题：基于柔性位置梯度反转层的ASR声学嵌入的设备内说话人匿名链接：https://arxiv.org/abs/2307.13343
作者：Md Asif Jalal,Pablo Peso Parada,Jisi Zhang,Karthikeyan Saravanan,Mete Ozay,Myoungji Han,Jung In Lee,Seokyeong Jung机构：Samsung Research, UK, AI R&D Group, Samsung Electronics, Suwon, South Korea备注：Proceedings of INTERSPEECH 2023摘要：由大规模AI模型服务的智能设备需要将用户数据传输到云端进行推理。对于语音应用，这意味着传送私人用户信息，例如，发言人身份我们的论文提出了一个隐私增强框架，目标说话人身份匿名化，同时保持语音识别的准确性，我们的下游任务-自动语音识别（ASR）。所提出的框架将灵活的基于梯度反转的说话人对抗层附加到ASR模型内的目标层，其中说话人对抗训练匿名化由目标层生成的声学嵌入以去除说话人身份。我们建议通过执行ASR模型的初始层来进行设备上部署，并将匿名嵌入传输到云，在云中执行模型的其余部分，同时保护隐私。实验结果表明，我们的方法有效地降低了说话人识别的相对准确率33%，并提高了ASR的性能，实现6.2%的相对字错误率（WER）减少。摘要：Smart devices serviced by large-scale AI models necessitates user data transfer to the cloud for inference. For speech applications, this means transferring private user information, e.g., speaker identity. Our paper proposes a privacy-enhancing framework that targets speaker identity anonymization while preserving speech recognition accuracy for our downstream task~-~Automatic Speech Recognition (ASR). The proposed framework attaches flexible gradient reversal based speaker adversarial layers to target layers within an ASR model, where speaker adversarial training anonymizes acoustic embeddings generated by the targeted layers to remove speaker identity. We propose on-device deployment by execution of initial layers of the ASR model, and transmitting anonymized embeddings to the cloud, where the rest of the model is executed while preserving privacy. Experimental results show that our method efficiently reduces speaker recognition relative accuracy by 33%, and improves ASR performance by achieving 6.2% relative Word Error Rate (WER) reduction.
【8】 IteraTTA: An interface for exploring both text prompts and audio priors  in generating music with text-to-audio models标题：IteraTTA：一个界面，用于在使用文本到音频模型生成音乐时探索文本提示和音频优先级链接：https://arxiv.org/abs/2307.13005
作者：Hiromu Yakura,Masataka Goto机构：University of Tsukuba, National Institute of Advanced Industrial Science, and Technology (AIST), Tsukuba, Japan备注：Accepted to the 24th International Society for Music Information Retrieval Conference (ISMIR 2023)摘要：最近的文本到音频生成技术具有允许新手用户自由地生成音乐音频的潜力。即使他们没有音乐知识，例如关于和弦进行和乐器，用户也可以尝试各种文本提示来生成音频。然而，与图像域相比，获得对可能的音乐音频的空间的清楚理解是困难的，因为用户不能同时收听所生成的音频的变化。因此，我们方便用户在探索不仅是文本提示，但也限制了文本到音频的音乐生成过程的音频先验。这种双侧探索使用户能够通过对它们的迭代比较来辨别不同文本提示和音频先验对生成结果的影响。我们开发的接口，IteraTTA，是专门设计来帮助用户在精炼文本提示和选择有利的音频先验从生成的音频。有了这个，用户可以逐步达到他们的松散指定的目标，同时了解和探索可能的结果的空间。我们的实施和讨论突出的设计考虑，特别是需要的文本到音频模型，以及如何交互技术可以有助于其有效性。摘要：Recent text-to-audio generation techniques have the potential to allow novice users to freely generate music audio. Even if they do not have musical knowledge, such as about chord progressions and instruments, users can try various text prompts to generate audio. However, compared to the image domain, gaining a clear understanding of the space of possible music audios is difficult because users cannot listen to the variations of the generated audios simultaneously. We therefore facilitate users in exploring not only text prompts but also audio priors that constrain the text-to-audio music generation process. This dual-sided exploration enables users to discern the impact of different text prompts and audio priors on the generation results through iterative comparison of them. Our developed interface, IteraTTA, is specifically designed to aid users in refining text prompts and selecting favorable audio priors from the generated audios. With this, users can progressively reach their loosely-specified goals while understanding and exploring the space of possible results. Our implementation and discussions highlight design considerations that are specifically required for text-to-audio models and how interaction techniques can contribute to their effectiveness.
eess.AS音频处理
【1】 On-Device Speaker Anonymization of Acoustic Embeddings for ASR based  onFlexible Location Gradient Reversal Layer标题：基于柔性位置梯度反转层的ASR声学嵌入的设备内说话人匿名链接：https://arxiv.org/abs/2307.13343
作者：Md Asif Jalal,Pablo Peso Parada,Jisi Zhang,Karthikeyan Saravanan,Mete Ozay,Myoungji Han,Jung In Lee,Seokyeong Jung机构：Samsung Research, UK, AI R&D Group, Samsung Electronics, Suwon, South Korea备注：Proceedings of INTERSPEECH 2023摘要：由大规模AI模型服务的智能设备需要将用户数据传输到云端进行推理。对于语音应用，这意味着传送私人用户信息，例如，发言人身份我们的论文提出了一个隐私增强框架，目标说话人身份匿名化，同时保持语音识别的准确性，我们的下游任务-自动语音识别（ASR）。所提出的框架将灵活的基于梯度反转的说话人对抗层附加到ASR模型内的目标层，其中说话人对抗训练匿名化由目标层生成的声学嵌入以去除说话人身份。我们建议通过执行ASR模型的初始层来进行设备上部署，并将匿名嵌入传输到云，在云中执行模型的其余部分，同时保护隐私。实验结果表明，我们的方法有效地降低了说话人识别的相对准确率33%，并提高了ASR的性能，实现6.2%的相对字错误率（WER）减少。摘要：Smart devices serviced by large-scale AI models necessitates user data transfer to the cloud for inference. For speech applications, this means transferring private user information, e.g., speaker identity. Our paper proposes a privacy-enhancing framework that targets speaker identity anonymization while preserving speech recognition accuracy for our downstream task~-~Automatic Speech Recognition (ASR). The proposed framework attaches flexible gradient reversal based speaker adversarial layers to target layers within an ASR model, where speaker adversarial training anonymizes acoustic embeddings generated by the targeted layers to remove speaker identity. We propose on-device deployment by execution of initial layers of the ASR model, and transmitting anonymized embeddings to the cloud, where the rest of the model is executed while preserving privacy. Experimental results show that our method efficiently reduces speaker recognition relative accuracy by 33%, and improves ASR performance by achieving 6.2% relative Word Error Rate (WER) reduction.
【2】 Adaptation of Whisper models to child speech recognition标题：耳语模型在儿童语音识别中的应用链接：https://arxiv.org/abs/2307.13008
作者：Rishabh Jain,Andrei Barcovschi,Mariam Yiwere,Peter Corcoran,Horia Cucu机构：School of Electrical and Electronics Engineering, University of Galway, Galway, Ireland,  Speech and Dialogue Research Laboratory, University Politehnica of Bucharest, Romania备注：Accepted in Interspeech 2023摘要：自动语音识别（ASR）系统通常难以转录儿童语音，这是由于缺乏准确训练儿童友好的ASR模型所需的大型儿童语音数据集。然而，有大量带注释的成人语音数据集用于创建多语言ASR模型，例如Whisper。我们的工作旨在探讨这些模型是否可以适应儿童语音，以提高儿童的ASR。此外，我们还比较了Whisper子适应与微调的自监督模型，如wav2vec2。我们表明，微调耳语对儿童语音产生显着改善ASR性能的儿童语音，相比非微调耳语模型。此外，利用已对儿童语音进行微调的自监督Wav2vec2模型优于Whisper微调。摘要：Automatic Speech Recognition (ASR) systems often struggle with transcribing child speech due to the lack of large child speech datasets required to accurately train child-friendly ASR models. However, there are huge amounts of annotated adult speech datasets which were used to create multilingual ASR models, such as Whisper. Our work aims to explore whether such models can be adapted to child speech to improve ASR for children. In addition, we compare Whisper child-adaptations with finetuned self-supervised models, such as wav2vec2. We demonstrate that finetuning Whisper on child speech yields significant improvements in ASR performance on child speech, compared to non finetuned Whisper models. Additionally, utilizing self-supervised Wav2vec2 models that have been finetuned on child speech outperforms Whisper finetuning.
【3】 IteraTTA: An interface for exploring both text prompts and audio priors  in generating music with text-to-audio models标题：IteraTTA：一个界面，用于在使用文本到音频模型生成音乐时探索文本提示和音频优先级链接：https://arxiv.org/abs/2307.13005
作者：Hiromu Yakura,Masataka Goto机构：University of Tsukuba, National Institute of Advanced Industrial Science, and Technology (AIST), Tsukuba, Japan备注：Accepted to the 24th International Society for Music Information Retrieval Conference (ISMIR 2023)摘要：最近的文本到音频生成技术具有允许新手用户自由地生成音乐音频的潜力。即使他们没有音乐知识，例如关于和弦进行和乐器，用户也可以尝试各种文本提示来生成音频。然而，与图像域相比，获得对可能的音乐音频的空间的清楚理解是困难的，因为用户不能同时收听所生成的音频的变化。因此，我们方便用户在探索不仅是文本提示，但也限制了文本到音频的音乐生成过程的音频先验。这种双侧探索使用户能够通过对它们的迭代比较来辨别不同文本提示和音频先验对生成结果的影响。我们开发的接口，IteraTTA，是专门设计来帮助用户在精炼文本提示和选择有利的音频先验从生成的音频。有了这个，用户可以逐步达到他们的松散指定的目标，同时了解和探索可能的结果的空间。我们的实施和讨论突出的设计考虑，特别是需要的文本到音频模型，以及如何交互技术可以有助于其有效性。摘要：Recent text-to-audio generation techniques have the potential to allow novice users to freely generate music audio. Even if they do not have musical knowledge, such as about chord progressions and instruments, users can try various text prompts to generate audio. However, compared to the image domain, gaining a clear understanding of the space of possible music audios is difficult because users cannot listen to the variations of the generated audios simultaneously. We therefore facilitate users in exploring not only text prompts but also audio priors that constrain the text-to-audio music generation process. This dual-sided exploration enables users to discern the impact of different text prompts and audio priors on the generation results through iterative comparison of them. Our developed interface, IteraTTA, is specifically designed to aid users in refining text prompts and selecting favorable audio priors from the generated audios. With this, users can progressively reach their loosely-specified goals while understanding and exploring the space of possible results. Our implementation and discussions highlight design considerations that are specifically required for text-to-audio models and how interaction techniques can contribute to their effectiveness.
【4】 Backdoor Attacks against Voice Recognition Systems: A Survey标题：针对语音识别系统的后门攻击：综述链接：https://arxiv.org/abs/2307.13643
作者：Baochen Yan,Jiahe Lan,Zheng Yan机构：Xidian University备注：33 pages, 7 figures摘要：语音识别系统（VRS）采用深度学习进行语音识别和说话人识别。它们已被广泛部署在各种现实应用中，从智能语音辅助到电话监控和生物识别认证。然而，先前的研究已经揭示了VRS的脆弱性后门攻击，这对VRS的安全性和隐私性构成了重大威胁。不幸的是，现有的文献缺乏一个彻底的审查这个问题。本文填补了这一研究空白，进行了全面的调查后门攻击VRS。我们首先介绍了VRS和后门攻击的概述，阐明了它们的基本知识。然后，我们提出了一套评估标准来评估后门攻击方法的性能。其次，从不同角度对虚拟现实后门攻击进行了分类，并分析了不同类型的后门攻击的特点。在此之后，我们全面审查现有的攻击方法，并分析其优点和缺点的基础上提出的标准。此外，我们回顾了经典的后门防御方法和通用的音频防御技术。然后，我们讨论了部署在虚拟现实系统的可行性。最后，我们指出了几个开放的问题，并进一步提出了未来的研究方向，以推动研究的虚拟现实系统的安全性。摘要：Voice Recognition Systems (VRSs) employ deep learning for speech recognition and speaker recognition. They have been widely deployed in various real-world applications, from intelligent voice assistance to telephony surveillance and biometric authentication. However, prior research has revealed the vulnerability of VRSs to backdoor attacks, which pose a significant threat to the security and privacy of VRSs. Unfortunately, existing literature lacks a thorough review on this topic. This paper fills this research gap by conducting a comprehensive survey on backdoor attacks against VRSs. We first present an overview of VRSs and backdoor attacks, elucidating their basic knowledge. Then we propose a set of evaluation criteria to assess the performance of backdoor attack methods. Next, we present a comprehensive taxonomy of backdoor attacks against VRSs from different perspectives and analyze the characteristic of different categories. After that, we comprehensively review existing attack methods and analyze their pros and cons based on the proposed criteria. Furthermore, we review classic backdoor defense methods and generic audio defense techniques. Then we discuss the feasibility of deploying them on VRSs. Finally, we figure out several open issues and further suggest future research directions to motivate the research of VRSs security.
【5】 Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals  using Self Supervised Speech Representations标题：使用自我监督语音表征的听障人士非侵入性可理解性预报器链接：https://arxiv.org/abs/2307.13423
作者：George Close,Thomas Hain,Stefan Goetze机构：Speech and Hearing Group, Department of Computer Science, University of Sheffield, United Kingdom备注：Submitted to ASRU 2023摘要：自监督语音表示（SSSR）已经成功地应用于许多语音处理任务，例如：作为用于语音质量（SQ）预测的特征提取器，其进而与用于具有正常或受损听力的用户的评估和训练语音增强系统相关。然而，确切的知识，为什么以及如何编码质量相关的信息，以及在这样的表示仍然知之甚少。在这项工作中，技术的非侵入性预测的SQ评级扩展到预测的听力受损的用户的可懂度。据发现，自我监督的表示是有用的输入功能，非侵入性的预测模型，实现更复杂的系统的竞争力的表现。对清晰度预测挑战1听众和增强系统性能的详细分析表明，可能需要更多的数据才能推广到未知系统和（听力受损）个人摘要：Self-supervised speech representations (SSSRs) have been successfully applied to a number of speech-processing tasks, e.g. as feature extractor for speech quality (SQ) prediction, which is, in turn, relevant for assessment and training speech enhancement systems for users with normal or impaired hearing. However, exact knowledge of why and how quality-related information is encoded well in such representations remains poorly understood. In this work, techniques for non-intrusive prediction of SQ ratings are extended to the prediction of intelligibility for hearing-impaired users. It is found that self-supervised representations are useful as input features to non-intrusive prediction models, achieving competitive performance to more complex systems. A detailed analysis of the performance depending on Clarity Prediction Challenge 1 listeners and enhancement systems indicates that more data might be needed to allow generalisation to unknown systems and (hearing-impaired) individuals
【6】 A Snoring Sound Dataset for Body Position Recognition: Collection,  Annotation, and Analysis标题：用于体位识别的鼾声数据集：采集、标注和分析链接：https://arxiv.org/abs/2307.13346
作者：Li Xiao,Xiuping Yang,Xinhong Li,Weiping Tu,Xiong Chen,Weiyan Yi,Jie Lin,Yuhong Yang,Yanzhen Ren机构：NERCMS, School of Computer Science, Wuhan University, Sleep Medicine Centre, Zhongnan Hospital of Wuhan University, School of Cyber Science and Engineering, Wuhan University备注：Accepted to INTERSPEECH 2023摘要：阻塞性睡眠呼吸暂停低通气综合征（OSAHS）是由上呼吸道阻塞引起的慢性呼吸障碍。打鼾是OSAHS的突出症状，并且先前的研究试图通过打鼾声音来识别上呼吸道的阻塞部位。尽管取得了一些进展，但由于睡眠体位对上呼吸道的影响，阻塞部位的分类在现实世界的临床环境中仍然具有挑战性。为了解决这一挑战，本文提出了一个基于打鼾的睡眠体位识别数据集（SSBPR），该数据集由7570个打鼾记录组成，包括六个不同的睡眠体位标签：仰卧、仰卧但左侧头、仰卧但右侧头、左侧卧、右侧卧和俯卧。实验结果表明，打鼾声音表现出某些声学特征，使得它们能够有效地用于在现实世界场景中的睡眠期间识别身体姿势。摘要：Obstructive Sleep Apnea-Hypopnea Syndrome (OSAHS) is a chronic breathing disorder caused by a blockage in the upper airways. Snoring is a prominent symptom of OSAHS, and previous studies have attempted to identify the obstruction site of the upper airways by snoring sounds. Despite some progress, the classification of the obstruction site remains challenging in real-world clinical settings due to the influence of sleep body position on upper airways. To address this challenge, this paper proposes a snore-based sleep body position recognition dataset (SSBPR) consisting of 7570 snoring recordings, which comprises six distinct labels for sleep body position: supine, supine but left lateral head, supine but right lateral head, left-side lying, right-side lying and prone. Experimental results show that snoring sounds exhibit certain acoustic features that enable their effective utilization for identifying body posture during sleep in real-world scenarios.
【7】 CQNV: A combination of coarsely quantized bitstream and neural vocoder  for low rate speech coding标题：CQNV：用于低速率语音编码的粗量化比特流和神经声码器的组合链接：https://arxiv.org/abs/2307.13295
作者：Youqiang Zheng,Li Xiao,Weiping Tu,Yuhong Yang,Xinmeng Xu机构：NERCMS, School of Computer Science, Wuhan University, China, Hubei Luojia Laboratory, China, Hubei Key Laboratory of Multimedia and Network Communication Engineering备注：Accepted by INTERSPEECH 2023摘要：最近，基于神经网络的语音编解码器已被证明比传统方法表现更好。然而，传统参数量化中的冗余在将传统编解码器与神经声码器组合的编解码器架构内是可见的。在本文中，我们提出了一种新的框架命名为CQNV，它结合了传统的参数化编解码器的粗量化参数，以降低比特率与神经声码器，以提高解码语音的质量。此外，我们引入了一个参数处理模块的神经声码器，以加强传统的语音编码参数的比特流的神经声码器的应用，进一步提高重建语音的质量。在实验中，主观和客观的评价证明了所提出的CQNV框架的有效性。具体来说，我们提出的方法可以实现更高质量的重建语音在1.1 kbps比天琴座和Encodec在3 kbps。摘要：Recently, speech codecs based on neural networks have proven to perform better than traditional methods. However, redundancy in traditional parameter quantization is visible within the codec architecture of combining the traditional codec with the neural vocoder. In this paper, we propose a novel framework named CQNV, which combines the coarsely quantized parameters of a traditional parametric codec to reduce the bitrate with a neural vocoder to improve the quality of the decoded speech. Furthermore, we introduce a parameters processing module into the neural vocoder to enhance the application of the bitstream of traditional speech coding parameters to the neural vocoder, further improving the reconstructed speech's quality. In the experiments, both subjective and objective evaluations demonstrate the effectiveness of the proposed CQNV framework. Specifically, our proposed method can achieve higher quality reconstructed speech at 1.1 kbps than Lyra and Encodec at 3 kbps.
【8】 Audio-aware Query-enhanced Transformer for Audio-Visual Segmentation标题：用于视听分割的音频感知查询增强型转换器链接：https://arxiv.org/abs/2307.13236
作者：Jinxiang Liu,Chen Ju,Chaofan Ma,Yanfeng Wang,Yu Wang,Ya Zhang机构：ShanghaiJiao Tong University备注：arXiv admin note: text overlap with arXiv:2305.11019摘要：视听分割（AVS）任务的目标是使用音频线索来分割视频帧中的发声对象。然而，目前基于融合的方法有性能的限制，由于小的卷积和视听功能的融合不足的接受域。为了克服这些问题，我们提出了一种新的\textbf{Au}dio-aware查询增强的\textbf{TR}ansformer（AuTR）来处理任务。与现有的方法不同，我们的方法引入了一个多模态Transformer架构，使深度融合和聚合的视听功能。此外，我们设计了一个音频感知的查询增强的Transformer解码器，明确地帮助模型专注于基于音频信号的精确定位的发声对象的分割，而忽略沉默但突出的对象。实验结果表明，该方法优于以往的方法，并表现出更好的泛化能力，在多声音和开放集的情况下。摘要：The goal of the audio-visual segmentation (AVS) task is to segment the sounding objects in the video frames using audio cues. However, current fusion-based methods have the performance limitations due to the small receptive field of convolution and inadequate fusion of audio-visual features. To overcome these issues, we propose a novel \textbf{Au}dio-aware query-enhanced \textbf{TR}ansformer (AuTR) to tackle the task. Unlike existing methods, our approach introduces a multimodal transformer architecture that enables deep fusion and aggregation of audio-visual features. Furthermore, we devise an audio-aware query-enhanced transformer decoder that explicitly helps the model focus on the segmentation of the pinpointed sounding objects based on audio signals, while disregarding silent yet salient objects. Experimental results show that our method outperforms previous methods and demonstrates better generalization ability in multi-sound and open-set scenarios.
【9】 Joint speech and overlap detection: a benchmark over multiple audio  setup and speech domains标题：联合语音和重叠检测：多个音频设置和语音域的基准链接：https://arxiv.org/abs/2307.13012
作者：Martin Lebourdais,Théo Mariotte,Marie Tahon,Anthony Larcher,Antoine Laurent,Silvio Montresor,Sylvain Meignier,Jean-Hugh Thomas机构：LIUM,LAUM UMR , IA-GS, Le Mans Universit´e摘要：语音活动和重叠语音检测（分别为VAD和OSD）是说话人日志化的关键预处理任务。最终的分割性能高度依赖于这些子任务的鲁棒性。最近的研究表明，VAD和OSD可以使用多类分类模型联合训练。然而，这些工作往往局限于一个特定的语音域，缺乏有关系统的泛化能力的信息。本文提出了一个完整的和新的基准不同的VAD和OSD模型，在多个音频设置（单/多通道）和语音域（例如，音频）。媒体、会议.）。我们的2/3类系统，结合了时间卷积网络与语音表示适应的设置，优于国家的最先进的结果。我们表明，这两个任务的联合训练提供了类似的性能，在F1分数两个专用的VAD和OSD系统，同时降低培训成本。这种独特的架构也可用于单声道和多声道语音处理。摘要：Voice activity and overlapped speech detection (respectively VAD and OSD) are key pre-processing tasks for speaker diarization. The final segmentation performance highly relies on the robustness of these sub-tasks. Recent studies have shown VAD and OSD can be trained jointly using a multi-class classification model. However, these works are often restricted to a specific speech domain, lacking information about the generalization capacities of the systems. This paper proposes a complete and new benchmark of different VAD and OSD models, on multiple audio setups (single/multi-channel) and speech domains (e.g. media, meeting...). Our 2/3-class systems, which combine a Temporal Convolutional Network with speech representations adapted to the setup, outperform state-of-the-art results. We show that the joint training of these two tasks offers similar performances in terms of F1-score to two dedicated VAD and OSD systems while reducing the training cost. This unique architecture can also be used for single and multichannel speech processing.
机器翻译，仅供参考