今日论文合集：cs.SD语音10篇，eess.AS音频处理10篇。
cs.SD语音
【1】 Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS
标题：端到端码转换TTS中段落处理的文本增强
链接：https://arxiv.org/abs/2210.11429
作者：Chunyu Qiang,Jianhua Tao,Ruibo Fu,Zhengqi Wen,Jiangyan Yi,Tao Wang,Shiming Wang机构：National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China, School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China备注：accepted in ISCSLP 2021摘要：现有的端到端语码转换文本语音转换（TTS）技术已经能够在单语者双语语料库中生成高质量的双语语音。当双语语料库中的说话者不同时，语码转换的自然性和一致性较差。我们提出的跨语言嵌入层结构使得不同语言中相似的音节具有相关性，从而提高了生成语音的自然度和一致性。在端到端的语码转换TTS中，段落文本合成时存在韵律不稳定的问题。本文提出的文本增强方法使输入包含韵律信息和句子级上下文信息，从而提高了段落文本的韵律稳定性。实验结果表明，所提方法在自然度、一致性和韵律稳定性等方面具有较好的效果。除普通话和英语外，我们还将这些方法应用于上海话和粤语语料库，证明了我们提出的方法可以扩展到其他语言来构建端到端的语码转换TTS系统。摘要：Current end-to-end code-switching Text-to-Speech (TTS) can already generate high quality two languages speech in the same utterance with single speaker bilingual corpora. When the speakers of the bilingual corpora are different, the naturalness and consistency of the code-switching TTS will be poor. The cross-lingual embedding layers structure we proposed makes similar syllables in different languages relevant, thus improving the naturalness and consistency of generated speech. In the end-to-end code-switching TTS, there exists problem of prosody instability when synthesizing paragraph text. The text enhancement method we proposed makes the input contain prosodic information and sentence-level context information, thus improving the prosody stability of paragraph text. Experimental results demonstrate the effectiveness of the proposed methods in the naturalness, consistency, and prosody stability. In addition to Mandarin and English, we also apply these methods to Shanghaiese and Cantonese corpora, proving that the methods we proposed can be extended to other languages to build end-to-end code-switching TTS system.
【2】 Play It Back: Iterative Attention for Audio Recognition
标题：回放：音频识别的迭代注意
链接：https://arxiv.org/abs/2210.11328
作者：Alexandros Stergiou,Dima Damen机构：Department of Electronics and Informatics, Vrije University of Brussels, Department of Computer Science, University of Bristol摘要：听觉认知的一个关键功能是随着时间的推移将特征声音与其相应的语义相关联。试图在细粒度音频类别之间进行区分的人经常重放相同的区分声音以增加他们的预测置信度。我们提出了一种端到端的基于注意力的体系结构，通过选择性重复来关注整个音频序列中最具区别性的声音。我们的模型最初使用完整的音频序列，并基于时隙注意力迭代地细化重放的时间段。在每次重放时，使用较小的跳跃长度重放所选择的段，该较小的跳跃长度表示这些段内的较高分辨率特征。我们证明，我们的方法可以在三个音频分类基准测试中始终如一地实现最先进的性能：音频设备、VGG声音和EPIC厨房-100。摘要：A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine-grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the-art performance across three audio-classification benchmarks: AudioSet, VGG-Sound, and EPIC-KITCHENS-100.
【3】 Robust One-Shot Singing Voice Conversion
标题：健壮的一次发声语音转换
链接：https://arxiv.org/abs/2210.11096
作者：Naoya Takahashi,Mayank Kumar Singh,Yuki Mitsufuji机构：Sony Group Corporation, Japan摘要：现有的许多歌唱声转换（SVC）的研究工作都需要对目标歌手的声音进行干净的录音训练。但往往很难提前收集，歌声往往会因混响和伴奏音乐而失真。在此工作中，我们提出一种鲁棒的单次SVC（ROSVC），即使在使用不到10s的参考语音的失真演唱语音上，也能鲁棒地执行任意到任意的SVC。为此，我们提出了两阶段训练方法Robustify。在第一阶段，基于生成式对抗网络，在干净数据上训练一种新的单次SVC模型，以保证高质量的转换。在第二阶段，将增强模块引入模型的编码器，以提高对特征空间中的失真的鲁棒性。实验结果表明，该方法对已见和未见歌手的检测性能均优于单次SVC基线检测，并大大提高了对失真的鲁棒性。摘要：Many existing works on singing voice conversion (SVC) require clean recordings of target singer's voice for training. However, it is often difficult to collect them in advance and singing voices are often distorted with reverb and accompaniment music. In this work, we propose robust one-shot SVC (ROSVC) that performs any-to-any SVC robustly even on such distorted singing voices using less than 10s of a reference voice. To this end, we propose two-stage training method called Robustify. In the first stage, a novel one-shot SVC model based on a generative adversarial network is trained on clean data to ensure high-quality conversion. In the second stage, enhancement modules are introduced to the encoders of the model to improve the robustness against distortions in the feature space. Experimental results show that the proposed method outperforms one-shot SVC baselines for both seen and unseen singers and greatly improves the robustness against the distortions.
【4】 Frequency of Interest-based Noise Attenuation Method to Improve Anomaly Detection Performance
标题：一种提高异常检测性能的基于兴趣频率的噪声抑制方法
链接：https://arxiv.org/abs/2210.11068
作者：YeongHyeon Park,Myung Jin Kim,Won Seok Park机构：SK Planet Co., Ltd., Seongnam, Rep. of Korea备注：5 pages, 4 figures, 4 tables摘要：在基于轮胎摩擦噪声的异常检测任务中，准确提取驾驶事件是最大化计算效率和异常检测性能的途径。该研究提出了一种简洁且非常有用的方法来提高事件提取的精度，该方法受到风噪声等额外噪声的阻碍，这些噪声由于其随机性而难以清楚地表征。该方法的核心是识别出与感兴趣频率相对应的路面摩擦声，并利用多个频率滤波器去除与感兴趣频率相反的特征。该方法能够最大限度地提高驾驶事件提取的精度，同时将异常检测性能平均提高了8.506%。因此，我们的结论是，我们的方法是一个实用的解决方案，适合于道路表面异常检测目的在户外边缘计算环境中。摘要：Accurately extracting driving events is the way to maximize computational efficiency and anomaly detection performance in the tire frictional nose-based anomaly detection task. This study proposes a concise and highly useful method for improving the precision of the event extraction that is hindered by extra noise such as wind noise, which is difficult to characterize clearly due to its randomness. The core of the proposed method is based on the identification of the road friction sound corresponding to the frequency of interest and removing the opposite characteristics with several frequency filters. Our method enables precision maximization of driving event extraction while improving anomaly detection performance by an average of 8.506%. Therefore, we conclude our method is a practical solution suitable for road surface anomaly detection purposes in outdoor edge computing environments.
【5】 Large-scale learning of generalised representations for speaker recognition
标题：说话人识别中广义表示的大规模学习
链接：https://arxiv.org/abs/2210.10985
作者：Jee-weon Jung,Hee-Soo Heo,Bong-Jin Lee,Jaesong Lee,Hye-jin Shim,Youngki Kwon,Joon Son Chung,Shinji Watanabe机构：Naver Corporation, South Korea ,School of Computing, University of Eastern Finland, Joensuu, Finland, Korea Advanced Institute of Science and Technology, South Korea, Carnegie Mellon University, Pittsburgh, PA, USA备注：5pages, 5 tables, submitted to ICASSP摘要：本研究的目的是开发一个可应用于不同情境的说话人辨识模型。我们假设两个组件应充分配置以构建这样的模型。首先，需要有适当的架构。我们探讨了几个最新的最先进的模型，包括ECAPA-TDNN和MFA-构象异构体，以及其他基线。第二，需要大量的数据。我们结合一些现有的数据集研究了几种新的训练数据配置。最广泛的配置包括超过87 k个扬声器的10.22k小时的讲话。采用四种评估协议来衡量训练好的模型在不同场景下的表现。通过实验发现，具有最小电感偏置的MFA-Conformer泛化效果最好。我们还表明，使用建议的大数据配置进行训练可提供更好的性能。观察到泛化能力的提高，其中四个评估协议的平均性能提高了20%以上。此外，我们还证明了这些模型的性能可以在增加容量时进一步提高。摘要：The objective of this work is to develop a speaker recognition model to be used in diverse scenarios. We hypothesise that two components should be adequately configured to build such a model. First, adequate architecture would be required. We explore several recent state-of-the-art models, including ECAPA-TDNN and MFA-Conformer, as well as other baselines. Second, a massive amount of data would be required. We investigate several new training data configurations combining a few existing datasets. The most extensive configuration includes over 87k speakers' 10.22k hours of speech. Four evaluation protocols are adopted to measure how the trained model performs in diverse scenarios. Through experiments, we find that MFA-Conformer with the least inductive bias generalises the best. We also show that training with proposed large data configurations gives better performance. A boost in generalisation is observed, where the average performance on four evaluation protocols improves by more than 20%. In addition, we also demonstrate that these models' performances can improve even further when increasing capacity.
【6】 G-Augment: Searching For The Meta-Structure Of Data Augmentation Policies For ASR
标题：G-Augment：寻找ASR数据增强策略的元结构
链接：https://arxiv.org/abs/2210.10879
作者：Gary Wang,Ekin D. Cubuk,Andrew Rosenberg,Shuyang Cheng,Ron J. Weiss,Bhuvana Ramabhadran,Pedro J. Moreno,Quoc V. Le,Daniel S. Park机构：Google Inc., †Pony.ai, ‡Meta Inc.备注：6 pages, accepted at SLT 2022摘要：数据增强是一种普遍存在的技术，用于为自动语音识别（ASR）训练提供鲁棒性。然而，即使ASR培训过程的大部分已经变得自动化并且更加“端到端”，数据扩充策略（使用什么扩充功能以及如何应用它们）仍然是手工制作的。本文提出了一种将扩展空间定义为有向无环图（DAG）并在此空间上搜索以优化扩展策略本身的技术--图-扩展。在CHiME-6和AMI上，我们证明了在相同的计算预算下，G-Augment策略比随机搜索SpecAugment策略在微调任务上有更好的性能. G-Augment还能够在CHiME-6评估集（30.7% WER）上建立新的最先进的ASR性能。我们进一步证明，与随机搜索的SpecAugment策略相比，G-Augment策略在从热启动到冷启动训练和模型大小方面显示出更好的传递特性。摘要：Data augmentation is a ubiquitous technique used to provide robustness to automatic speech recognition (ASR) training. However, even as so much of the ASR training process has become automated and more "end-to-end", the data augmentation policy (what augmentation functions to use, and how to apply them) remains hand-crafted. We present Graph-Augment, a technique to define the augmentation space as directed acyclic graphs (DAGs) and search over this space to optimize the augmentation policy itself. We show that given the same computational budget, policies produced by G-Augment are able to perform better than SpecAugment policies obtained by random search on fine-tuning tasks on CHiME-6 and AMI. G-Augment is also able to establish a new state-of-the-art ASR performance on the CHiME-6 evaluation set (30.7% WER). We further demonstrate that G-Augment policies show better transfer properties across warm-start to cold-start training and model size compared to random-searched SpecAugment policies.
【7】 Modeling Animal Vocalizations through Synthesizers
标题：通过合成器模拟动物发声
链接：https://arxiv.org/abs/2210.10857
作者：Masato Hagiwara,Maddie Cusimano,Jen-Yu Liu机构：Earth Species Project, TorchSynth, L(x, y), Encoder, (optional), Optimizer, Parameters, Target (cat), Prediction (cat), Noise, Pitch, Env.摘要：对真实世界的声音进行建模是机器学习和许多其他领域（包括人类语音处理和生物声学）创造性使用的一个基本问题。基于转换器的生成模型和一些先前的工作（例如，DDSP）已知可产生逼真的声音，尽管它们具有有限的控制且难以解释。作为替代方案，我们的目标是使用模块化合成器，即，一种用于模拟非音乐声音的作曲参数电子乐器。然而，在给定目标声音的情况下推断合成器参数，即，参数推断任务对于一般声音来说不是微不足道的，并且过去的研究通常集中在音乐声音上。在这项工作中，我们优化了一个来自TorchSynth的可区分合成器，以便建模、仿真和创造性地生成动物发声。我们比较了一系列用于推断其参数的优化方法，从基于梯度的搜索到遗传算法，然后演示了如何控制和解释用于建模非音乐声音的参数。摘要：Modeling real-world sound is a fundamental problem in the creative use of machine learning and many other fields, including human speech processing and bioacoustics. Transformer-based generative models and some prior work (e.g., DDSP) are known to produce realistic sound, although they have limited control and are hard to interpret. As an alternative, we aim to use modular synthesizers, i.e., compositional, parametric electronic musical instruments, for modeling non-music sounds. However, inferring synthesizer parameters given a target sound, i.e., the parameter inference task, is not trivial for general sounds, and past research has typically focused on musical sound. In this work, we optimize a differentiable synthesizer from TorchSynth in order to model, emulate, and creatively generate animal vocalizations. We compare an array of optimization methods, from gradient-based search to genetic algorithms, for inferring its parameters, and then demonstrate how one can control and interpret the parameters for modeling non-music sounds.
【8】 Model-matching Principle Applied to the Design of an Array-based All-neural Binaural Rendering System for Audio Telepresence
标题：模型匹配原理在阵列式全神经双耳临场感绘制系统设计中的应用
链接：https://arxiv.org/abs/2210.11123
作者：Yicheng Hsu,Chenghumg Ma,Mingsian R. Bai机构：Department of Power Mechanical Engineering, National Tsing Hua University, Taiwan, Department of Electrical Engineering, National Tsing Hua University, Taiwan备注：submitted to ICASSP 2023摘要：网真旨在为近端用户创造一种身临其境但又虚拟的远端视听场景体验。本文提出了一种基于阵列的双耳渲染系统，该系统将阵列麦克风信号转换为经头相关传递函数（HRTF）滤波的输出信号，用于头戴式耳机渲染。该方法基于模型匹配原理（MMP），与传统的定位-波束形成-HRTF滤波（LBH）方法相比，具有更自然的沉浸感。基于MMP的渲染系统可以通过多通道逆滤波（MIF）和多通道深度滤波（MDF）来实现。在本研究中，我们采用了MDF方法，并使用LBH和MIF作为基线。在生成双耳输出之前，全神经系统联合地捕获空间信息（空间呈现）、保留环境声音（增强）以及减少噪声（增强）。通过客观和主观测试，将所提出的远程临场感系统与两种基准系统进行了比较。摘要：Telepresence aims to create an immersive but virtual experience of the far-end audio and visual scene for users at the near-end. In this contribution, we propose an array-based binaural rendering system that converts the array microphone signals into the head-related transfer function (HRTF)-filtered output signals for headphone-rendering. The proposed approach is formulated on the basis of a model-matching principle (MMP) and is capable of delivering more natural immersiveness than the conventional localization-beamforming-HRTF filtering (LBH) approach. The MMP-based rendering system can be realized via multichannel inverse filtering (MIF) and multichannel deep filtering (MDF). In this study, we adopted the MDF approach and used the LBH as well as MIF as the baselines. The all-neural system jointly captures the spatial information (spatial rendering), preserves ambient sound (enhancement), and reduces noise (enhancement), prior to generating binaural outputs. Objective and subjective tests are employed to compare the proposed telepresence system with two baselines.
【9】 Speech Dereverberation with a Reverberation Time Shortening Target
标题：以缩短混响时间为目标的语音去混响
链接：https://arxiv.org/abs/2210.11089
作者：Rui Zhou,Wenye Zhu,Xiaofei Li机构：Westlake University & Westlake Institute for Advanced Study, Hangzhou, China, Zhejiang University, Hangzhou, China备注：submitted to ICASSP 2023. arXiv admin note: substantial text overlap with arXiv:2204.08765摘要：本文提出了一种基于混响时间缩短的语音去混响学习目标。去混响的学习目标通常被设置为直接路径语音或可选地具有一些早期反射。这种类型的目标会突然截断混响，因此可能不适合网络训练。该RTS目标在抑制混响的同时保持了混响的指数衰减特性，简化了网络训练，从而降低了预测误差引起的信号失真。此外，本论文还对我们先前提出的全子网语音去噪网络进行了实验研究，以适应语音去混响。实验结果表明，RTS比直接路径语音和早期反射语音具有更好的混响抑制效果和信号失真抑制效果，是一种更适合的学习目标。FullSubNet能够实现出色的去混响性能。摘要：This work proposes a new learning target based on reverberation time shortening (RTS) for speech dereverberation. The learning target for dereverberation is usually set as the direct-path speech or optionally with some early reflections. This type of target suddenly truncates the reverberation, and thus it may not be suitable for network training. The proposed RTS target suppresses reverberation and meanwhile maintains the exponential decaying property of reverberation, which will ease the network training, and thus reduce signal distortion caused by the prediction error. Moreover, this work experimentally study to adapt our previously proposed FullSubNet speech denoising network to speech dereverberation. Experiments show that RTS is a more suitable learning target than direct-path speech and early reflections, in terms of better suppressing reverberation and signal distortion. FullSubNet is able to achieve outstanding dereverberation performance.
【10】 DisC-VC: Disentangled and F0-Controllable Neural Voice Conversion
标题：Disk-VC：解缠和F0可控的神经语音转换
链接：https://arxiv.org/abs/2210.11059
作者：Chihiro Watanabe,Hirokazu Kameoka机构：NTT Communication Science Laboratories,-, Morinosato Wakamiya, Atsugi-shi, Kanagawa Pref., -, Japan摘要：语音转换是对给定话语的非语言特征进行转换的任务。由于语音的自然度强烈地依赖于其音调模式，因此在一些应用中，希望在改变说话者身份的同时保持原始的上升/下降音调模式。一些现有的方法通过使用源-滤波器模型或开发将F0模式作为模型输入的神经网络来解决该问题。虽然后一种方法与前一种方法相比可以获得相对较高的声音质量，但是在其训练过程中没有考虑目标和生成的F0模式之间的差异。本文提出了一种新的基于变分自动编码器的语音转换模型，并在模型中加入了辅助网络，以保证转换结果正确反映指定的F0/音色信息。通过客观和主观评价，验证了该方法的有效性。摘要：Voice conversion is a task to convert a non-linguistic feature of a given utterance. Since naturalness of speech strongly depends on its pitch pattern, in some applications, it would be desirable to keep the original rise/fall pitch pattern while changing the speaker identity. Some of the existing methods address this problem by either using a source-filter model or developing a neural network that takes an F0 pattern as input to the model. Although the latter approach can achieve relatively high sound quality compared to the former one, there is no consideration for discrepancy between the target and generated F0 patterns in its training process. In this paper, we propose a new variational-autoencoder-based voice conversion model accompanied by an auxiliary network, which ensures that the conversion result correctly reflects the specified F0/timbre information. We show the effectiveness of the proposed method by objective and subjective evaluations.
【1】 Model-matching Principle Applied to the Design of an Array-based All-neural Binaural Rendering System for Audio Telepresence
标题：模型匹配原理在阵列式全神经双耳临场感绘制系统设计中的应用
链接：https://arxiv.org/abs/2210.11123
* 与cs.SD语音【8】为同一篇
作者：Yicheng Hsu,Chenghumg Ma,Mingsian R. Bai机构：Department of Power Mechanical Engineering, National Tsing Hua University, Taiwan, Department of Electrical Engineering, National Tsing Hua University, Taiwan备注：submitted to ICASSP 2023摘要：网真旨在为近端用户创造一种身临其境但又虚拟的远端视听场景体验。本文提出了一种基于阵列的双耳渲染系统，该系统将阵列麦克风信号转换为经头相关传递函数（HRTF）滤波的输出信号，用于头戴式耳机渲染。该方法基于模型匹配原理（MMP），与传统的定位-波束形成-HRTF滤波（LBH）方法相比，具有更自然的沉浸感。基于MMP的渲染系统可以通过多通道逆滤波（MIF）和多通道深度滤波（MDF）来实现。在本研究中，我们采用了MDF方法，并使用LBH和MIF作为基线。在生成双耳输出之前，全神经系统联合地捕获空间信息（空间呈现）、保留环境声音（增强）以及减少噪声（增强）。通过客观和主观测试，将所提出的远程临场感系统与两种基准系统进行了比较。摘要：Telepresence aims to create an immersive but virtual experience of the far-end audio and visual scene for users at the near-end. In this contribution, we propose an array-based binaural rendering system that converts the array microphone signals into the head-related transfer function (HRTF)-filtered output signals for headphone-rendering. The proposed approach is formulated on the basis of a model-matching principle (MMP) and is capable of delivering more natural immersiveness than the conventional localization-beamforming-HRTF filtering (LBH) approach. The MMP-based rendering system can be realized via multichannel inverse filtering (MIF) and multichannel deep filtering (MDF). In this study, we adopted the MDF approach and used the LBH as well as MIF as the baselines. The all-neural system jointly captures the spatial information (spatial rendering), preserves ambient sound (enhancement), and reduces noise (enhancement), prior to generating binaural outputs. Objective and subjective tests are employed to compare the proposed telepresence system with two baselines.
【2】 Speech Dereverberation with a Reverberation Time Shortening Target
标题：以缩短混响时间为目标的语音去混响
链接：https://arxiv.org/abs/2210.11089
* 与cs.SD语音【9】为同一篇
作者：Rui Zhou,Wenye Zhu,Xiaofei Li机构：Westlake University & Westlake Institute for Advanced Study, Hangzhou, China, Zhejiang University, Hangzhou, China备注：submitted to ICASSP 2023. arXiv admin note: substantial text overlap with arXiv:2204.08765摘要：本文提出了一种基于混响时间缩短的语音去混响学习目标。去混响的学习目标通常被设置为直接路径语音或可选地具有一些早期反射。这种类型的目标会突然截断混响，因此可能不适合网络训练。该RTS目标在抑制混响的同时保持了混响的指数衰减特性，简化了网络训练，从而降低了预测误差引起的信号失真。此外，本论文还对我们先前提出的全子网语音去噪网络进行了实验研究，以适应语音去混响。实验结果表明，RTS比直接路径语音和早期反射语音具有更好的混响抑制效果和信号失真抑制效果，是一种更适合的学习目标。FullSubNet能够实现出色的去混响性能。摘要：This work proposes a new learning target based on reverberation time shortening (RTS) for speech dereverberation. The learning target for dereverberation is usually set as the direct-path speech or optionally with some early reflections. This type of target suddenly truncates the reverberation, and thus it may not be suitable for network training. The proposed RTS target suppresses reverberation and meanwhile maintains the exponential decaying property of reverberation, which will ease the network training, and thus reduce signal distortion caused by the prediction error. Moreover, this work experimentally study to adapt our previously proposed FullSubNet speech denoising network to speech dereverberation. Experiments show that RTS is a more suitable learning target than direct-path speech and early reflections, in terms of better suppressing reverberation and signal distortion. FullSubNet is able to achieve outstanding dereverberation performance.
【3】 DisC-VC: Disentangled and F0-Controllable Neural Voice Conversion
标题：Disk-VC：解缠和F0可控的神经语音转换
链接：https://arxiv.org/abs/2210.11059
* 与cs.SD语音【10】为同一篇
作者：Chihiro Watanabe,Hirokazu Kameoka机构：NTT Communication Science Laboratories,-, Morinosato Wakamiya, Atsugi-shi, Kanagawa Pref., -, Japan摘要：语音转换是对给定话语的非语言特征进行转换的任务。由于语音的自然度强烈地依赖于其音调模式，因此在一些应用中，希望在改变说话者身份的同时保持原始的上升/下降音调模式。一些现有的方法通过使用源-滤波器模型或开发将F0模式作为模型输入的神经网络来解决该问题。虽然后一种方法与前一种方法相比可以获得相对较高的声音质量，但是在其训练过程中没有考虑目标和生成的F0模式之间的差异。本文提出了一种新的基于变分自动编码器的语音转换模型，并在模型中加入了辅助网络，以保证转换结果正确反映指定的F0/音色信息。通过客观和主观评价，验证了该方法的有效性。摘要：Voice conversion is a task to convert a non-linguistic feature of a given utterance. Since naturalness of speech strongly depends on its pitch pattern, in some applications, it would be desirable to keep the original rise/fall pitch pattern while changing the speaker identity. Some of the existing methods address this problem by either using a source-filter model or developing a neural network that takes an F0 pattern as input to the model. Although the latter approach can achieve relatively high sound quality compared to the former one, there is no consideration for discrepancy between the target and generated F0 patterns in its training process. In this paper, we propose a new variational-autoencoder-based voice conversion model accompanied by an auxiliary network, which ensures that the conversion result correctly reflects the specified F0/timbre information. We show the effectiveness of the proposed method by objective and subjective evaluations.
【4】 Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS
标题：端到端码转换TTS中段落处理的文本增强
链接：https://arxiv.org/abs/2210.11429
* 与cs.SD语音【1】为同一篇
作者：Chunyu Qiang,Jianhua Tao,Ruibo Fu,Zhengqi Wen,Jiangyan Yi,Tao Wang,Shiming Wang机构：National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China, School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China备注：accepted in ISCSLP 2021摘要：现有的端到端语码转换文本语音转换（TTS）技术已经能够在单语者双语语料库中生成高质量的双语语音。当双语语料库中的说话者不同时，语码转换的自然性和一致性较差。我们提出的跨语言嵌入层结构使得不同语言中相似的音节具有相关性，从而提高了生成语音的自然度和一致性。在端到端的语码转换TTS中，段落文本合成时存在韵律不稳定的问题。本文提出的文本增强方法使输入包含韵律信息和句子级上下文信息，从而提高了段落文本的韵律稳定性。实验结果表明，所提方法在自然度、一致性和韵律稳定性等方面具有较好的效果。除普通话和英语外，我们还将这些方法应用于上海话和粤语语料库，证明了我们提出的方法可以扩展到其他语言来构建端到端的语码转换TTS系统。摘要：Current end-to-end code-switching Text-to-Speech (TTS) can already generate high quality two languages speech in the same utterance with single speaker bilingual corpora. When the speakers of the bilingual corpora are different, the naturalness and consistency of the code-switching TTS will be poor. The cross-lingual embedding layers structure we proposed makes similar syllables in different languages relevant, thus improving the naturalness and consistency of generated speech. In the end-to-end code-switching TTS, there exists problem of prosody instability when synthesizing paragraph text. The text enhancement method we proposed makes the input contain prosodic information and sentence-level context information, thus improving the prosody stability of paragraph text. Experimental results demonstrate the effectiveness of the proposed methods in the naturalness, consistency, and prosody stability. In addition to Mandarin and English, we also apply these methods to Shanghaiese and Cantonese corpora, proving that the methods we proposed can be extended to other languages to build end-to-end code-switching TTS system.
【5】 Play It Back: Iterative Attention for Audio Recognition
标题：回放：音频识别的迭代注意
链接：https://arxiv.org/abs/2210.11328
* 与cs.SD语音【2】为同一篇
作者：Alexandros Stergiou,Dima Damen机构：Department of Electronics and Informatics, Vrije University of Brussels, Department of Computer Science, University of Bristol摘要：听觉认知的一个关键功能是随着时间的推移将特征声音与其相应的语义相关联。试图在细粒度音频类别之间进行区分的人经常重放相同的区分声音以增加他们的预测置信度。我们提出了一种端到端的基于注意力的体系结构，通过选择性重复来关注整个音频序列中最具区别性的声音。我们的模型最初使用完整的音频序列，并基于时隙注意力迭代地细化重放的时间段。在每次重放时，使用较小的跳跃长度重放所选择的段，该较小的跳跃长度表示这些段内的较高分辨率特征。我们证明，我们的方法可以在三个音频分类基准测试中始终如一地实现最先进的性能：音频设备、VGG声音和EPIC厨房-100。摘要：A key function of auditory cognition is the association of characteristic sounds with their corresponding semantics over time. Humans attempting to discriminate between fine-grained audio categories, often replay the same discriminative sounds to increase their prediction confidence. We propose an end-to-end attention-based architecture that through selective repetition attends over the most discriminative sounds across the audio sequence. Our model initially uses the full audio sequence and iteratively refines the temporal segments replayed based on slot attention. At each playback, the selected segments are replayed using a smaller hop length which represents higher resolution features within these segments. We show that our method can consistently achieve state-of-the-art performance across three audio-classification benchmarks: AudioSet, VGG-Sound, and EPIC-KITCHENS-100.
【6】 Robust One-Shot Singing Voice Conversion
标题：健壮的一次发声语音转换
链接：https://arxiv.org/abs/2210.11096
* 与cs.SD语音【3】为同一篇
作者：Naoya Takahashi,Mayank Kumar Singh,Yuki Mitsufuji机构：Sony Group Corporation, Japan摘要：现有的许多歌唱声转换（SVC）的研究工作都需要对目标歌手的声音进行干净的录音训练。但往往很难提前收集，歌声往往会因混响和伴奏音乐而失真。在此工作中，我们提出一种鲁棒的单次SVC（ROSVC），即使在使用不到10s的参考语音的失真演唱语音上，也能鲁棒地执行任意到任意的SVC。为此，我们提出了两阶段训练方法Robustify。在第一阶段，基于生成式对抗网络，在干净数据上训练一种新的单次SVC模型，以保证高质量的转换。在第二阶段，将增强模块引入模型的编码器，以提高对特征空间中的失真的鲁棒性。实验结果表明，该方法对已见和未见歌手的检测性能均优于单次SVC基线检测，并大大提高了对失真的鲁棒性。摘要：Many existing works on singing voice conversion (SVC) require clean recordings of target singer's voice for training. However, it is often difficult to collect them in advance and singing voices are often distorted with reverb and accompaniment music. In this work, we propose robust one-shot SVC (ROSVC) that performs any-to-any SVC robustly even on such distorted singing voices using less than 10s of a reference voice. To this end, we propose two-stage training method called Robustify. In the first stage, a novel one-shot SVC model based on a generative adversarial network is trained on clean data to ensure high-quality conversion. In the second stage, enhancement modules are introduced to the encoders of the model to improve the robustness against distortions in the feature space. Experimental results show that the proposed method outperforms one-shot SVC baselines for both seen and unseen singers and greatly improves the robustness against the distortions.
【7】 Frequency of Interest-based Noise Attenuation Method to Improve Anomaly Detection Performance
标题：一种提高异常检测性能的基于兴趣频率的噪声抑制方法
链接：https://arxiv.org/abs/2210.11068
* 与cs.SD语音【4】为同一篇
作者：YeongHyeon Park,Myung Jin Kim,Won Seok Park机构：SK Planet Co., Ltd., Seongnam, Rep. of Korea备注：5 pages, 4 figures, 4 tables摘要：在基于轮胎摩擦噪声的异常检测任务中，准确提取驾驶事件是最大化计算效率和异常检测性能的途径。该研究提出了一种简洁且非常有用的方法来提高事件提取的精度，该方法受到风噪声等额外噪声的阻碍，这些噪声由于其随机性而难以清楚地表征。该方法的核心是识别出与感兴趣频率相对应的路面摩擦声，并利用多个频率滤波器去除与感兴趣频率相反的特征。该方法能够最大限度地提高驾驶事件提取的精度，同时将异常检测性能平均提高了8.506%。因此，我们的结论是，我们的方法是一个实用的解决方案，适合于道路表面异常检测目的在户外边缘计算环境中。摘要：Accurately extracting driving events is the way to maximize computational efficiency and anomaly detection performance in the tire frictional nose-based anomaly detection task. This study proposes a concise and highly useful method for improving the precision of the event extraction that is hindered by extra noise such as wind noise, which is difficult to characterize clearly due to its randomness. The core of the proposed method is based on the identification of the road friction sound corresponding to the frequency of interest and removing the opposite characteristics with several frequency filters. Our method enables precision maximization of driving event extraction while improving anomaly detection performance by an average of 8.506%. Therefore, we conclude our method is a practical solution suitable for road surface anomaly detection purposes in outdoor edge computing environments.
【8】 Large-scale learning of generalised representations for speaker recognition
标题：说话人识别中广义表示的大规模学习
链接：https://arxiv.org/abs/2210.10985
* 与cs.SD语音【5】为同一篇
作者：Jee-weon Jung,Hee-Soo Heo,Bong-Jin Lee,Jaesong Lee,Hye-jin Shim,Youngki Kwon,Joon Son Chung,Shinji Watanabe机构：Naver Corporation, South Korea ,School of Computing, University of Eastern Finland, Joensuu, Finland, Korea Advanced Institute of Science and Technology, South Korea, Carnegie Mellon University, Pittsburgh, PA, USA备注：5pages, 5 tables, submitted to ICASSP摘要：本研究的目的是开发一个可应用于不同情境的说话人辨识模型。我们假设两个组件应充分配置以构建这样的模型。首先，需要有适当的架构。我们探讨了几个最新的最先进的模型，包括ECAPA-TDNN和MFA-构象异构体，以及其他基线。第二，需要大量的数据。我们结合一些现有的数据集研究了几种新的训练数据配置。最广泛的配置包括超过87 k个扬声器的10.22k小时的讲话。采用四种评估协议来衡量训练好的模型在不同场景下的表现。通过实验发现，具有最小电感偏置的MFA-Conformer泛化效果最好。我们还表明，使用建议的大数据配置进行训练可提供更好的性能。观察到泛化能力的提高，其中四个评估协议的平均性能提高了20%以上。此外，我们还证明了这些模型的性能可以在增加容量时进一步提高。摘要：The objective of this work is to develop a speaker recognition model to be used in diverse scenarios. We hypothesise that two components should be adequately configured to build such a model. First, adequate architecture would be required. We explore several recent state-of-the-art models, including ECAPA-TDNN and MFA-Conformer, as well as other baselines. Second, a massive amount of data would be required. We investigate several new training data configurations combining a few existing datasets. The most extensive configuration includes over 87k speakers' 10.22k hours of speech. Four evaluation protocols are adopted to measure how the trained model performs in diverse scenarios. Through experiments, we find that MFA-Conformer with the least inductive bias generalises the best. We also show that training with proposed large data configurations gives better performance. A boost in generalisation is observed, where the average performance on four evaluation protocols improves by more than 20%. In addition, we also demonstrate that these models' performances can improve even further when increasing capacity.
【9】 G-Augment: Searching For The Meta-Structure Of Data Augmentation Policies For ASR
标题：G-Augment：寻找ASR数据增强策略的元结构
链接：https://arxiv.org/abs/2210.10879
* 与cs.SD语音【6】为同一篇
作者：Gary Wang,Ekin D. Cubuk,Andrew Rosenberg,Shuyang Cheng,Ron J. Weiss,Bhuvana Ramabhadran,Pedro J. Moreno,Quoc V. Le,Daniel S. Park机构：Google Inc., †Pony.ai, ‡Meta Inc.备注：6 pages, accepted at SLT 2022摘要：数据增强是一种普遍存在的技术，用于为自动语音识别（ASR）训练提供鲁棒性。然而，即使ASR培训过程的大部分已经变得自动化并且更加“端到端”，数据扩充策略（使用什么扩充功能以及如何应用它们）仍然是手工制作的。本文提出了一种将扩展空间定义为有向无环图（DAG）并在此空间上搜索以优化扩展策略本身的技术--图-扩展。在CHiME-6和AMI上，我们证明了在相同的计算预算下，G-Augment策略比随机搜索SpecAugment策略在微调任务上有更好的性能. G-Augment还能够在CHiME-6评估集（30.7% WER）上建立新的最先进的ASR性能。我们进一步证明，与随机搜索的SpecAugment策略相比，G-Augment策略在从热启动到冷启动训练和模型大小方面显示出更好的传递特性。摘要：Data augmentation is a ubiquitous technique used to provide robustness to automatic speech recognition (ASR) training. However, even as so much of the ASR training process has become automated and more "end-to-end", the data augmentation policy (what augmentation functions to use, and how to apply them) remains hand-crafted. We present Graph-Augment, a technique to define the augmentation space as directed acyclic graphs (DAGs) and search over this space to optimize the augmentation policy itself. We show that given the same computational budget, policies produced by G-Augment are able to perform better than SpecAugment policies obtained by random search on fine-tuning tasks on CHiME-6 and AMI. G-Augment is also able to establish a new state-of-the-art ASR performance on the CHiME-6 evaluation set (30.7% WER). We further demonstrate that G-Augment policies show better transfer properties across warm-start to cold-start training and model size compared to random-searched SpecAugment policies.
【10】 Modeling Animal Vocalizations through Synthesizers
标题：通过合成器模拟动物发声
链接：https://arxiv.org/abs/2210.10857
* 与cs.SD语音【7】为同一篇
作者：Masato Hagiwara,Maddie Cusimano,Jen-Yu Liu机构：Earth Species Project, TorchSynth, L(x, y), Encoder, (optional), Optimizer, Parameters, Target (cat), Prediction (cat), Noise, Pitch, Env.摘要：对真实世界的声音进行建模是机器学习和许多其他领域（包括人类语音处理和生物声学）创造性使用的一个基本问题。基于转换器的生成模型和一些先前的工作（例如，DDSP）已知可产生逼真的声音，尽管它们具有有限的控制且难以解释。作为替代方案，我们的目标是使用模块化合成器，即，一种用于模拟非音乐声音的作曲参数电子乐器。然而，在给定目标声音的情况下推断合成器参数，即，参数推断任务对于一般声音来说不是微不足道的，并且过去的研究通常集中在音乐声音上。在这项工作中，我们优化了一个来自TorchSynth的可区分合成器，以便建模、仿真和创造性地生成动物发声。我们比较了一系列用于推断其参数的优化方法，从基于梯度的搜索到遗传算法，然后演示了如何控制和解释用于建模非音乐声音的参数。摘要：Modeling real-world sound is a fundamental problem in the creative use of machine learning and many other fields, including human speech processing and bioacoustics. Transformer-based generative models and some prior work (e.g., DDSP) are known to produce realistic sound, although they have limited control and are hard to interpret. As an alternative, we aim to use modular synthesizers, i.e., compositional, parametric electronic musical instruments, for modeling non-music sounds. However, inferring synthesizer parameters given a target sound, i.e., the parameter inference task, is not trivial for general sounds, and past research has typically focused on musical sound. In this work, we optimize a differentiable synthesizer from TorchSynth in order to model, emulate, and creatively generate animal vocalizations. We compare an array of optimization methods, from gradient-based search to genetic algorithms, for inferring its parameters, and then demonstrate how one can control and interpret the parameters for modeling non-music sounds.
机器翻译，仅供参考
每日语音速递