今天跟大家分享一篇语音相关的论文合集：cs.SD语音6篇，eess.AS音频处理9篇。
【1】 An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
标题：深度学习时代的情感语音合成与转换综述
链接：https://arxiv.org/abs/2210.03538
作者：Andreas Triantafyllopoulos,Björn W. Schuller,Gökçe İymen,Metin Sezgin,Xiangheng He,Zijiang Yang,Panagiotis Tzirakis,Shuo Liu,Silvan Mertes,Elisabeth André,Ruibo Fu,Jianhua Tao机构：Member, IEEE备注：Submitted to the Proceedings of IEEE摘要：语音是人类交流的基本方式，其合成一直是人机交互研究的核心内容。近年来，机器已经成功地掌握了生成人类可以理解的语音的艺术。但是，话语的语言内容只包含其意义的一部分。情感，或表达力，有能力把言语变成一种媒介，能够传达亲密的想法，感觉和情绪-这是至关重要的方面，从事和自然的人际沟通。虽然赋予合成话语以表现力的目标迄今为止仍然难以实现，但是随着文本到语音合成的最新进展，在情感语音合成和转换领域中也正在进行范式转变。深度学习是人工智能领域最新进展的技术基础，是这些努力的先锋。在本综述中，我们概述了正在进行的趋势，并总结了最新的方法，试图提供一个全面的概述这个令人兴奋的领域。摘要：Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human-computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. But the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions -- aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesised utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology which underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In the present overview, we outline ongoing trends and summarise state-of-the-art approaches in an attempt to provide a comprehensive overview of this exciting field.
【2】 Model-based estimation of in-car-communication feedback applied to speech zone detection
标题：基于模型的车内通信反馈估计在语音区域检测中的应用
链接：https://arxiv.org/abs/2210.03363
作者：Kaspar Müller,Simon Doclo,Jan Østergaard,Tobias Wolff机构：University of Oldenburg, Department of Medical Physics and Acoustics and, Cluster of Excellence Hearing,all, Oldenburg, Germany, Aalborg University, Department of Electronic Systems, Aalborg, Denmark备注：5 pages, submitted to International Workshop on Acoustic Signal Enhancement (IWAENC), Bamberg, Germany, 2022摘要：现代汽车提供了多功能工具来增强语音通信。虽然车内通信（ICC）系统旨在通过经由车内的扬声器回放期望的语音来增强乘客之间的通信，但是这些扬声器信号可能干扰免提电话和自动语音识别所需的语音增强系统。本文重点研究了语音区域检测，即检测车内哪个乘客在说话，这是语音增强系统的一个关键组成部分。提出了一种基于模型的反馈估计方法，以提高语音区域检测对ICC反馈的鲁棒性。具体地，由于区域检测系统通常不能访问ICC扬声器信号，因此所提出的方法基于扬声器和麦克风之间的自由场传播模型以及ICC增益来估计来自所观察的麦克风信号的反馈信号。我们提出了一种在短时傅里叶变换域中使用卷积传递函数的有效递归实现。仿真结果表明，该方法在保证语音区域检测结果鲁棒性的前提下，可以将ICC增益提高约6dB。摘要：Modern cars provide versatile tools to enhance speech communication. While an in-car communication (ICC) system aims at enhancing communication between the passengers by playing back desired speech via loudspeakers in the car, these loudspeaker signals may disturb a speech enhancement system required for hands-free telephony and automatic speech recognition. In this paper, we focus on speech zone detection, i.e. detecting which passenger in the car is speaking, which is a crucial component of the speech enhancement system. We propose a model-based feedback estimation method to improve robustness of speech zone detection against ICC feedback. Specifically, since the zone detection system typically does not have access to the ICC loudspeaker signals, the proposed method estimates the feedback signal from the observed microphone signals based on a free-field propagation model between the loudspeakers and the microphones as well as the ICC gain. We propose an efficient recursive implementation in the short-time Fourier transform domain using convolutive transfer functions. A realistic simulation study indicates that the proposed method allows to increase the ICC gain by about 6dB while still achieving robust speech zone detection results.
【3】 The PerspectiveLiberator -- an upmixing 6DoF rendering plugin for single-perspective Ambisonic room impulse responses
标题：PerspectiveLiberator--一个混搭的6DoF渲染插件，用于单视角两性房间脉冲响应
链接：https://arxiv.org/abs/2210.03360
作者：Kaspar Müller,Franz Zotter机构：Institute of Electronic Music and Acoustics, University of Music and Performing Arts Graz, Austria备注：None摘要：目前，虚拟现实界面允许用户虚拟地在六自由度（6DoF）上改变视角，并且与视觉部分一致，声学视角需要交互地更新。具有动态头部旋转的单视角渲染已经非常可靠地与上混的一阶立体声房间脉冲响应（ASDM、SIRR等）一起工作。这个贡献提供了一个插件，通过实时透视外推从测量的透视中释放虚拟透视：解放者。该插件允许在两种不同的方向分辨率增强算法（ASDM、4DE）之间进行选择。对于其基于卷积的6DoF渲染的主要任务，该插件检测并定位早期Ambisonic房间脉冲响应中的突出方向声音事件，并使用方向、到达时间和电平对它们进行重新编码，以适应虚拟听众的可变视角。漫射残差在方向分辨率上被增强，但是保持不受平移运动的影响，以尽可能多地保留原始房间印象。摘要：Nowadays, virtual reality interfaces allow the user to change perspectives in six degrees of freedom (6DoF) virtually, and consistently with the visual part, the acoustic perspective needs to be updated interactively. Single-perspective rendering with dynamic head rotation already works quite reliably with upmixed first-order Ambisonic room impulse responses (ASDM, SIRR, etc.). This contribution presents a plugin to free the virtual perspective from the measured one by real-time perspective extrapolation: The PerspectiveLiberator. The plugin permits selecting between two different algorithms for directional resolution enhancement (ASDM, 4DE). And for its main task of convolution-based 6DoF rendering, the plugin detects and localizes prominent directional sound events in the early Ambisonic room impulse response and re-encodes them with direction, time of arrival, and level adapted to the variable perspective of the virtual listener. The diffuse residual is enhanced in directional resolution but remains unaffected by translatory movement to preserve as much of the original room impression as possible.
【4】 Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognition
标题：基于换能器的自动语音识别领域自适应过程中的损伤控制
链接：https://arxiv.org/abs/2210.03255
作者：Somshubra Majumdar,Shantanu Acharya,Vitaly Lavrukhin,Boris Ginsburg备注：To appear in Proc. SLT 2022, Jan 09-12, 2023, Doha, Qatar摘要：自动语音识别模型经常被修改以提高它们在新领域中的准确性。模型适应新领域的一个潜在缺点是灾难性遗忘，其中原始领域上的字错误率显著降低。本文讨论了当我们希望同时使自动语音识别模型适应新的领域和限制在原始领域上的准确度下降而不访问原始训练数据集时的情况。我们提出了几种技术，如有限的训练策略和正则化适配器模块，用于传感器编码器、预测和连接器网络。我们将这些方法应用于Google语音命令以及英国和爱尔兰英语方言语音数据集，并在新目标域上获得强结果，同时限制了原始域上的降级。摘要：Automatic speech recognition models are often adapted to improve their accuracy in a new domain. A potential drawback of model adaptation to new domains is catastrophic forgetting, where the Word Error Rate on the original domain is significantly degraded. This paper addresses the situation when we want to simultaneously adapt automatic speech recognition models to a new domain and limit the degradation of accuracy on the original domain without access to the original training dataset. We propose several techniques such as a limited training strategy and regularized adapter modules for the Transducer encoder, prediction, and joiner network. We apply these methods to the Google Speech Commands and to the UK and Ireland English Dialect speech data set and obtain strong results on the new target domain while limiting the degradation on the original domain.
【5】 Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarization
标题：单通道和多通道端到端神经网络的互学习
链接：https://arxiv.org/abs/2210.03459
作者：Shota Horiguchi,Yuki Takashima,Shinji Watanabe,Paola Garcia机构：Paola Garc´ıa, Hitachi, Ltd., Japan, Carnegie Mellon University, USA, Johns Hopkins University, USA备注：Accepted to IEEE SLT 2022摘要：由于多通道语音处理的高性能，我们可以使用来自多通道模型的输出作为教师标签来训练具有知识提取的单通道模型。相反，还已知的是，通过在训练期间将单通道语音数据与多通道语音数据混合或者通过将其用于模型预训练，单通道语音数据可以有益于多通道模型。本文以说话人日记化为研究对象，提出了交替进行上述双向知识转移的方法。我们首先介绍了一个端到端的神经diarization模型，它可以处理单通道和多通道输入。使用该模型，我们交替地进行i）从多通道模型到单通道模型的知识提取和ii）从提取的单通道模型到多通道模型的微调。在两个说话人数据上的实验结果表明，该方法可以同时提高单通道和多通道说话人的diarization性能。摘要：Due to the high performance of multi-channel speech processing, we can use the outputs from a multi-channel model as teacher labels when training a single-channel model with knowledge distillation. To the contrary, it is also known that single-channel speech data can benefit multi-channel models by mixing it with multi-channel speech data during training or by using it for model pretraining. This paper focuses on speaker diarization and proposes to conduct the above bi-directional knowledge transfer alternately. We first introduce an end-to-end neural diarization model that can handle both single- and multi-channel inputs. Using this model, we alternately conduct i) knowledge distillation from a multi-channel model to a single-channel model and ii) finetuning from the distilled single-channel model to a multi-channel model. Experimental results on two-speaker data show that the proposed method mutually improved single- and multi-channel speaker diarization performances.
【6】 Exploration of A Self-Supervised Speech Model: A Study on Emotional Corpora
标题：基于情感语料库的自监督语音模型研究
链接：https://arxiv.org/abs/2210.02595
作者：Yuanchao Li,Yumnah Mohamied,Peter Bell,Catherine Lai机构：Centre for Speech Technology Research, University of Edinburgh摘要：在过去的几年中，自监督语音模型已经快速发展，并且已经证明在各种下游任务中使用是可行的。最近的一些工作已经开始研究这些模型的特征，但许多问题还没有得到充分解决。本文以情感语料库为研究对象，探讨了一种流行的自我监督模型--wav 2 vec 2.0。通过一系列的定量分析，我们主要论证了：1）wav2vec2.0似乎丢弃了对于单词识别目的不太有用的副语言信息; 2）对于情感识别，单独来自中间层的表示的性能与通过层平均得到的表示的性能一样好，而最终层在某些情况下导致最差的性能; 3）当前的自监督模型可能不是利用非词汇特征的下游任务的最佳解决方案。我们的工作提供了新的发现，将有助于未来的研究在这一领域和理论基础的使用现有的模型。摘要：Self-supervised speech models have grown fast during the past few years and have proven feasible for use in various downstream tasks. Some recent work has started to look at the characteristics of these models, yet many concerns have not been fully addressed. In this work, we conduct a study on emotional corpora to explore a popular self-supervised model -- wav2vec 2.0. Via a set of quantitative analysis, we mainly demonstrate that: 1) wav2vec 2.0 appears to discard paralinguistic information that is less useful for word recognition purposes; 2) for emotion recognition, representations from the middle layer alone perform as well as those derived from layer averaging, while the final layer results in the worst performance in some cases; 3) current self-supervised models may not be the optimal solution for downstream tasks that make use of non-lexical features. Our work provides novel findings that will aid future research in this area and theoretical basis for the use of existing models.
【1】 Synthetic Voice Detection and Audio Splicing Detection using SE-Res2Net-Conformer Architecture
标题：基于SE-Res2Net-Conform架构的合成语音检测和音频拼接检测
链接：https://arxiv.org/abs/2210.03581
作者：Lei Wang,Benedict Yeoh,Jun Wah Ng机构：KLASS Engineering and Solutions Pte Ltd备注：Accepted by the 13th International Symposium on Chinese Spoken Language Processing (ISCSLP 2022)摘要：为了欺骗互联网用户和人工智能（AI）技术（如语音认证），已经生成了合成语音和拼接音频片段。现有的研究工作将欺骗对策视为二元分类问题：真实与欺骗。本文通过引入Conformer块来扩展现有的Res 2Net，以进一步挖掘声学特征的局部模式。在ASVspoof 2019数据库上的实验结果表明，所提出的SE-Res 2Net-Conformer体系结构能够提高逻辑访问场景下的欺骗对抗性能。 此外，本文还提出了对现有的音频拼接检测问题进行重新表述。与识别完整的拼接段相比，检测拼接段的边界更有用。此外，深度学习方法可以用于解决该问题，这与以前的信号处理技术不同。摘要：Synthetic voice and splicing audio clips have been generated to spoof Internet users and artificial intelligence (AI) technologies such as voice authentication. Existing research work treats spoofing countermeasures as a binary classification problem: bonafide vs. spoof. This paper extends the existing Res2Net by involving the recent Conformer block to further exploit the local patterns on acoustic features. Experimental results on ASVspoof 2019 database show that the proposed SE-Res2Net-Conformer architecture is able to improve the spoofing countermeasures performance for the logical access scenario. In addition, this paper also proposes to re-formulate the existing audio splicing detection problem. Instead of identifying the complete splicing segments, it is more useful to detect the boundaries of the spliced segments. Moreover, a deep learning approach can be used to solve the problem, which is different from the previous signal processing techniques.
【2】 Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarization
标题：单通道和多通道端到端神经网络的互学习
链接：https://arxiv.org/abs/2210.03459
* 与cs.SD语音【5】为同一篇
作者：Shota Horiguchi,Yuki Takashima,Shinji Watanabe,Paola Garcia机构：Paola Garc´ıa, Hitachi, Ltd., Japan, Carnegie Mellon University, USA, Johns Hopkins University, USA备注：Accepted to IEEE SLT 2022摘要：由于多通道语音处理的高性能，我们可以使用来自多通道模型的输出作为教师标签来训练具有知识提取的单通道模型。相反，还已知的是，通过在训练期间将单通道语音数据与多通道语音数据混合或者通过将其用于模型预训练，单通道语音数据可以有益于多通道模型。本文以说话人日记化为研究对象，提出了交替进行上述双向知识转移的方法。我们首先介绍了一个端到端的神经diarization模型，它可以处理单通道和多通道输入。使用该模型，我们交替地进行i）从多通道模型到单通道模型的知识提取和ii）从提取的单通道模型到多通道模型的微调。在两个说话人数据上的实验结果表明，该方法可以同时提高单通道和多通道说话人的diarization性能。摘要：Due to the high performance of multi-channel speech processing, we can use the outputs from a multi-channel model as teacher labels when training a single-channel model with knowledge distillation. To the contrary, it is also known that single-channel speech data can benefit multi-channel models by mixing it with multi-channel speech data during training or by using it for model pretraining. This paper focuses on speaker diarization and proposes to conduct the above bi-directional knowledge transfer alternately. We first introduce an end-to-end neural diarization model that can handle both single- and multi-channel inputs. Using this model, we alternately conduct i) knowledge distillation from a multi-channel model to a single-channel model and ii) finetuning from the distilled single-channel model to a multi-channel model. Experimental results on two-speaker data show that the proposed method mutually improved single- and multi-channel speaker diarization performances.
【3】 SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training
标题：SpeechUT：基于编解码器的语音-文本预训练中的隐藏单元桥接语音和文本
链接：https://arxiv.org/abs/2210.03730
作者：Ziqiang Zhang,Long Zhou,Junyi Ao,Shujie Liu,Lirong Dai,Jinyu Li,Furu Wei机构：University of Science and Technology of China, Microsoft, The Chinese University of Hong Kong, Shenzhen备注：14 pages, accepted by EMNLP 2022摘要：单模态预训练的快速发展促使研究者更加关注跨模态预训练方法。本文提出了一种统一模态的语音单元文本预训练模型SpeechUT，通过共享单元编码器将语音编码器和文本解码器的表示连接起来。利用隐藏单元作为对齐语音和文本的接口，我们可以将语音-文本模型分解为语音-单元模型和单元-文本模型，这两个模型可以分别用不成对的语音和文本数据进行联合预训练。我们提出的SpeechUT在自动语音识别（ASR）和语音翻译（ST）任务上进行了微调和评估。实验结果表明，SpeechUT在较强的基线上有了很大的改进，在LibriSpeech ASR和Must-C ST两个任务上都达到了最佳性能。为了更好地理解所提议的SpeechUT，进行了详细的分析。代码和预先训练的模型可在www.example.com上获得https://aka.ms/SpeechUT。摘要：The rapid development of single-modal pre-training has prompted researchers to pay more attention to cross-modal pre-training methods. In this paper, we propose a unified-modal speech-unit-text pre-training model, SpeechUT, to connect the representations of a speech encoder and a text decoder with a shared unit encoder. Leveraging hidden-unit as an interface to align speech and text, we can decompose the speech-to-text model into a speech-to-unit model and a unit-to-text model, which can be jointly pre-trained with unpaired speech and text data respectively. Our proposed SpeechUT is fine-tuned and evaluated on automatic speech recognition (ASR) and speech translation (ST) tasks. Experimental results show that SpeechUT gets substantial improvements over strong baselines, and achieves state-of-the-art performance on both the LibriSpeech ASR and MuST-C ST tasks. To better understand the proposed SpeechUT, detailed analyses are conducted. The code and pre-trained models are available at https://aka.ms/SpeechUT.
【4】 Pronunciation Modeling of Foreign Words for Mandarin ASR by Considering the Effect of Language Transfer
标题：考虑语言迁移影响的汉语ASR外来词发音建模
链接：https://arxiv.org/abs/2210.03603
机构：Institute for Infocomm Research, Singapore备注：Published by INTERSPEECH 2014摘要：外语词识别是自动语音识别的难点之一。研究发现，说话者的外语发音受其母语知识的影响，这种现象被称为语言迁移效应。本文主要研究自动语音识别中语言迁移的语音效应。提出了一套将英语单词转换为汉语语音表示的词法规则。以这种方式，普通话词典可以通过包括英语单词来扩充。因此，普通话ASR系统变得能够识别英语单词，而无需重新训练或重新估计声学模型参数。使用由所提出的规则导出的词典，在不损害纯汉语语音的准确性的情况下，提高了汉语英语混合语音的ASR性能。本文提出的词汇规则具有一定的推广性，可以直接应用于未出现的英语单词。摘要：One of the challenges in automatic speech recognition is foreign words recognition. It is observed that a speaker's pronunciation of a foreign word is influenced by his native language knowledge, and such phenomenon is known as the effect of language transfer. This paper focuses on examining the phonetic effect of language transfer in automatic speech recognition. A set of lexical rules is proposed to convert an English word into Mandarin phonetic representation. In this way, a Mandarin lexicon can be augmented by including English words. Hence, the Mandarin ASR system becomes capable to recognize English words without retraining or re-estimation of the acoustic model parameters. Using the lexicon that derived from the proposed rules, the ASR performance of Mandarin English mixed speech is improved without harming the accuracy of Mandarin only speech. The proposed lexical rules are generalized and they can be directly applied to unseen English words.
【5】 Cloud-based Automatic Speech Recognition Systems for Southeast Asian Languages
标题：基于云的东南亚语种自动语音识别系统
链接：https://arxiv.org/abs/2210.03580
作者：Lei Wang,Rong Tong,Cheung Chi Leung,Sunil Sivadas,Chongjia Ni,Bin Ma机构：Human Language Department, Institute for Infocomm Research, Singapore备注：Published by the 2017 IEEE International Conference on Orange Technologies (ICOT 2017)摘要：本文全面介绍了我们的东南亚语言自动语音识别系统。由于目前对这类区域语言的工作还不多，在建立系统之前，应解决一些困难：本文以印尼语和泰语为例，说明了如何收集建立ASR系统所需的各种资源的策略。摘要：This paper provides an overall introduction of our Automatic Speech Recognition (ASR) systems for Southeast Asian languages. As not much existing work has been carried out on such regional languages, a few difficulties should be addressed before building the systems: limitation on speech and text resources, lack of linguistic knowledge, etc. This work takes Bahasa Indonesia and Thai as examples to illustrate the strategies of collecting various resources required for building ASR systems.
【6】 Model-based estimation of in-car-communication feedback applied to speech zone detection
标题：基于模型的车内通信反馈估计在语音区域检测中的应用
链接：https://arxiv.org/abs/2210.03363
* 与cs.SD语音【2】为同一篇
作者：Kaspar Müller,Simon Doclo,Jan Østergaard,Tobias Wolff机构：University of Oldenburg, Department of Medical Physics and Acoustics and, Cluster of Excellence Hearing,all, Oldenburg, Germany, Aalborg University, Department of Electronic Systems, Aalborg, Denmark备注：5 pages, submitted to International Workshop on Acoustic Signal Enhancement (IWAENC), Bamberg, Germany, 2022摘要：现代汽车提供了多功能工具来增强语音通信。虽然车内通信（ICC）系统旨在通过经由车内的扬声器回放期望的语音来增强乘客之间的通信，但是这些扬声器信号可能干扰免提电话和自动语音识别所需的语音增强系统。本文重点研究了语音区域检测，即检测车内哪个乘客在说话，这是语音增强系统的一个关键组成部分。提出了一种基于模型的反馈估计方法，以提高语音区域检测对ICC反馈的鲁棒性。具体地，由于区域检测系统通常不能访问ICC扬声器信号，因此所提出的方法基于扬声器和麦克风之间的自由场传播模型以及ICC增益来估计来自所观察的麦克风信号的反馈信号。我们提出了一种在短时傅里叶变换域中使用卷积传递函数的有效递归实现。仿真结果表明，该方法在保证语音区域检测结果鲁棒性的前提下，可以将ICC增益提高约6dB。摘要：Modern cars provide versatile tools to enhance speech communication. While an in-car communication (ICC) system aims at enhancing communication between the passengers by playing back desired speech via loudspeakers in the car, these loudspeaker signals may disturb a speech enhancement system required for hands-free telephony and automatic speech recognition. In this paper, we focus on speech zone detection, i.e. detecting which passenger in the car is speaking, which is a crucial component of the speech enhancement system. We propose a model-based feedback estimation method to improve robustness of speech zone detection against ICC feedback. Specifically, since the zone detection system typically does not have access to the ICC loudspeaker signals, the proposed method estimates the feedback signal from the observed microphone signals based on a free-field propagation model between the loudspeakers and the microphones as well as the ICC gain. We propose an efficient recursive implementation in the short-time Fourier transform domain using convolutive transfer functions. A realistic simulation study indicates that the proposed method allows to increase the ICC gain by about 6dB while still achieving robust speech zone detection results.
【7】 The PerspectiveLiberator -- an upmixing 6DoF rendering plugin for single-perspective Ambisonic room impulse responses
标题：PerspectiveLiberator--一个混搭的6DoF渲染插件，用于单视角两性房间脉冲响应
链接：https://arxiv.org/abs/2210.03360
* 与cs.SD语音【3】为同一篇
作者：Kaspar Müller,Franz Zotter机构：Institute of Electronic Music and Acoustics, University of Music and Performing Arts Graz, Austria备注：None摘要：目前，虚拟现实界面允许用户虚拟地在六自由度（6DoF）上改变视角，并且与视觉部分一致，声学视角需要交互地更新。具有动态头部旋转的单视角渲染已经非常可靠地与上混的一阶立体声房间脉冲响应（ASDM、SIRR等）一起工作。这个贡献提供了一个插件，通过实时透视外推从测量的透视中释放虚拟透视：解放者。该插件允许在两种不同的方向分辨率增强算法（ASDM、4DE）之间进行选择。对于其基于卷积的6DoF渲染的主要任务，该插件检测并定位早期Ambisonic房间脉冲响应中的突出方向声音事件，并使用方向、到达时间和电平对它们进行重新编码，以适应虚拟听众的可变视角。漫射残差在方向分辨率上被增强，但是保持不受平移运动的影响，以尽可能多地保留原始房间印象。摘要：Nowadays, virtual reality interfaces allow the user to change perspectives in six degrees of freedom (6DoF) virtually, and consistently with the visual part, the acoustic perspective needs to be updated interactively. Single-perspective rendering with dynamic head rotation already works quite reliably with upmixed first-order Ambisonic room impulse responses (ASDM, SIRR, etc.). This contribution presents a plugin to free the virtual perspective from the measured one by real-time perspective extrapolation: The PerspectiveLiberator. The plugin permits selecting between two different algorithms for directional resolution enhancement (ASDM, 4DE). And for its main task of convolution-based 6DoF rendering, the plugin detects and localizes prominent directional sound events in the early Ambisonic room impulse response and re-encodes them with direction, time of arrival, and level adapted to the variable perspective of the virtual listener. The diffuse residual is enhanced in directional resolution but remains unaffected by translatory movement to preserve as much of the original room impression as possible.
【8】 Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognition
标题：基于换能器的自动语音识别领域自适应过程中的损伤控制
链接：https://arxiv.org/abs/2210.03255
* 与cs.SD语音【4】为同一篇
作者：Somshubra Majumdar,Shantanu Acharya,Vitaly Lavrukhin,Boris Ginsburg备注：To appear in Proc. SLT 2022, Jan 09-12, 2023, Doha, Qatar摘要：自动语音识别模型经常被修改以提高它们在新领域中的准确性。模型适应新领域的一个潜在缺点是灾难性遗忘，其中原始领域上的字错误率显著降低。本文讨论了当我们希望同时使自动语音识别模型适应新的领域和限制在原始领域上的准确度下降而不访问原始训练数据集时的情况。我们提出了几种技术，如有限的训练策略和正则化适配器模块，用于传感器编码器、预测和连接器网络。我们将这些方法应用于Google语音命令以及英国和爱尔兰英语方言语音数据集，并在新目标域上获得强结果，同时限制了原始域上的降级。摘要：Automatic speech recognition models are often adapted to improve their accuracy in a new domain. A potential drawback of model adaptation to new domains is catastrophic forgetting, where the Word Error Rate on the original domain is significantly degraded. This paper addresses the situation when we want to simultaneously adapt automatic speech recognition models to a new domain and limit the degradation of accuracy on the original domain without access to the original training dataset. We propose several techniques such as a limited training strategy and regularized adapter modules for the Transducer encoder, prediction, and joiner network. We apply these methods to the Google Speech Commands and to the UK and Ireland English Dialect speech data set and obtain strong results on the new target domain while limiting the degradation on the original domain.
【9】 Robotic Speech Synthesis: Perspectives on Interactions, Scenarios, and Ethics
标题：机器人语音合成：交互、情景和伦理的视角
链接：https://arxiv.org/abs/2203.09599
作者：Yuanchao Li,Catherine Lai机构：School of Informatics, University of Edinburgh, UK, School of Philosophy, Psychology & Language Sciences备注：Accepted for the HRI 2022 Workshop "Robo-Identity: Exploring Artificial Identity and Emotion via Speech Interactions" at HRI 2022, 7 March 2022摘要：近年来，许多工作已经研究了对话机器人执行特定任务的可行性，如医疗和采访。伴随这一发展而来的是一个实际问题：我们应该如何合成机器人的声音来满足不同情况的需要？本文从三个角度对这一问题进行了探讨：1）合成非语言的和面向交互的语音信号的困难，特别是反向信道; 2）机器人语音合成的场景分类; 3）机器人声音设计的伦理问题。本文将介绍相关文献的研究结果和我们的前期工作，试图引起人机交互研究者的注意，以设计出更好的会话机器人。摘要：In recent years, many works have investigated the feasibility of conversational robots for performing specific tasks, such as healthcare and interview. Along with this development comes a practical issue: how should we synthesize robotic voices to meet the needs of different situations? In this paper, we discuss this issue from three perspectives: 1) the difficulties of synthesizing non-verbal and interaction-oriented speech signals, particularly backchannels; 2) the scenario classification for robotic voice synthesis; 3) the ethical issues regarding the design of robot voice for its emotion and identity. We present the findings of relevant literature and our prior work, trying to bring the attention of human-robot interaction researchers to design better conversational robots in the future.
机器翻译，仅供参考
每日语音速递