今天跟大家分享一篇语音相关的论文合集：cs.SD语音5篇，eess.AS音频处理4篇。
【1】 ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognition
标题：ConvRNN-T：用于流语音识别的卷积增广递归神经网络换能器
链接：https://arxiv.org/abs/2209.14868
作者：Martin Radfar,Rohit Barnwal,Rupak Vignesh Swaminathan,Feng-Ju Chang,Grant P. Strimel,Nathan Susanj,Athanasios Mouchtaris机构：Alexa Machine Learning, Amazon, USA备注：This paper was presented in Interspeech 2022摘要：递归神经网络传感器（RNN-T）是一种重要的流式端到端（E2 E）ASR技术。在RNN-T中，声编码器通常由LSTM的堆叠组成。最近，作为LSTM层的替代，Conformer架构被引入，其中RNN-T的编码器被修改的Transformer编码器代替，该编码器由前端和关注层之间的卷积层组成。本文提出了一种新的流式ASR模型--卷积增强递归神经网络转换器（Convronational Augmented Recurrent Neural Network Transducers，ConvRNN-T），它是在基于LSTM的RNN-T的基础上增加了一个由局部和全局上下文CNN编码器组成的卷积前端。ConvRNN-T利用因果1-D卷积层、挤压和激励、膨胀和残差块来向LSTM层提供全局和局部音频上下文表示。我们显示ConvRNN-T在Librispeech和内部数据上的性能优于RNN-T、Conformer和ContextNet。此外，与Conformer相比，ConvRNN-T的计算复杂度更低。ConvRNN-T的高精度和低占用空间使其成为设备上流式ASR技术的一个有前途的候选产品。摘要：The recurrent neural network transducer (RNN-T) is a prominent streaming end-to-end (E2E) ASR technology. In RNN-T, the acoustic encoder commonly consists of stacks of LSTMs. Very recently, as an alternative to LSTM layers, the Conformer architecture was introduced where the encoder of RNN-T is replaced with a modified Transformer encoder composed of convolutional layers at the frontend and between attention layers. In this paper, we introduce a new streaming ASR model, Convolutional Augmented Recurrent Neural Network Transducers (ConvRNN-T) in which we augment the LSTM-based RNN-T with a novel convolutional frontend consisting of local and global context CNN encoders. ConvRNN-T takes advantage of causal 1-D convolutional layers, squeeze-and-excitation, dilation, and residual blocks to provide both global and local audio context representation to LSTM layers. We show ConvRNN-T outperforms RNN-T, Conformer, and ContextNet on Librispeech and in-house data. In addition, ConvRNN-T offers less computational complexity compared to Conformer. ConvRNN-T's superior accuracy along with its low footprint make it a promising candidate for on-device streaming ASR technologies.
【2】 Classification of Vocal Bursts for ACII 2022 A-VB-Type Competition using Convolutional Network Networks and Deep Acoustic Embeddings
标题：基于卷积网络和深声嵌入的ACII2022A-VB型比赛发声群分类
链接：https://arxiv.org/abs/2209.14842
作者：Muhammad Shehram Shah Syed,Zafi Sherhan Syed,Abbas Syed机构：Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro, Pakistan, Department of Telecommunication Engineering, Department of Electronic Engineering备注：Report for our submission to the ACII 2022 Affective Vocal Bursts (A-VB) Competition摘要：本报告简要描述了我们为ACII 2022情感爆发性发声（A-VB）竞赛的爆发性发声类型分类任务提出的解决方案。我们试验了两种方法，作为我们手头任务解决方案的一部分。第一种是基于Mel谱图训练的卷积神经网络，第二种是基于来自预先训练的wav 2 vec 2模型的深度声学嵌入的平均池。我们的最佳模型实现了测试分区的0.5190的未加权平均召回率（UAR），而机会水平UAR为0.1250，基线为0.4172。因此，比激发基线提高约20%。本文中报告的结果证明了我们提出的解决AV-B类型分类任务的方法的有效性。摘要：This report provides a brief description of our proposed solution for the Vocal Burst Type classification task of the ACII 2022 Affective Vocal Bursts (A-VB) Competition. We experimented with two approaches as part of our solution for the task at hand. The first of which is based on convolutional neural networks trained on Mel Spectrograms, and the second is based on average pooling of deep acoustic embeddings from a pretrained wav2vec2 model. Our best performing model achieves an unweighted average recall (UAR) of 0.5190 for the test partition, compared to the chance-level UAR of 0.1250 and a baseline of 0.4172. Thus, an improvement of around 20% over the challenge baseline. The results reported in this document demonstrate the efficacy of our proposed approaches to solve the AV-B Type Classification task.
【3】 The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling
标题：室群生成器：通过产生式建模获得无限高质量的MIR数据
链接：https://arxiv.org/abs/2209.14458
作者：Yusong Wu,Josh Gardner,Ethan Manilow,Ian Simon,Curtis Hawthorne,Jesse Engel机构：Université de Montréal, Mila, University of Washington, Google Research, Brain Team摘要：数据是现代机器学习系统的生命线，包括音乐信息检索（MIR）中的数据。然而，长期以来，MIR一直被小数据集和不可靠的标签所困扰。在这项工作中，我们提出了打破这一瓶颈使用创成式建模。通过将音符生成模型（Coconet在Bach Chorales上训练）与室内乐合奏的结构化合成模型（MIDI-DDSP在URMP上训练）流水线化，我们展示了一个能够产生无限数量的具有丰富注释的真实合唱音乐的系统，包括混音、词干、MIDI、音符级演奏属性（断奏、颤音等），以及甚至细粒度的合成参数（音调、幅度等）。我们称这个系统为腔室集合生成器（CEG），并使用它从四个不同的腔室集合（CocoChorales）中生成大的合唱数据集。我们证明了使用我们的方法生成的数据改进了音乐转录和源分离的最先进模型，我们发布了系统和数据集，作为MIR社区未来工作的开源基础。摘要：Data is the lifeblood of modern machine learning systems, including for those in Music Information Retrieval (MIR). However, MIR has long been mired by small datasets and unreliable labels. In this work, we propose to break this bottleneck using generative modeling. By pipelining a generative model of notes (Coconet trained on Bach Chorales) with a structured synthesis model of chamber ensembles (MIDI-DDSP trained on URMP), we demonstrate a system capable of producing unlimited amounts of realistic chorale music with rich annotations including mixes, stems, MIDI, note-level performance attributes (staccato, vibrato, etc.), and even fine-grained synthesis parameters (pitch, amplitude, etc.). We call this system the Chamber Ensemble Generator (CEG), and use it to generate a large dataset of chorales from four different chamber ensembles (CocoChorales). We demonstrate that data generated using our approach improves state-of-the-art models for music transcription and source separation, and we release both the system and the dataset as an open-source foundation for future work in the MIR community.
【4】 Audio Barlow Twins: Self-Supervised Audio Representation Learning
标题：Audio Barlow双胞胎：自我监督的音频表征学习
链接：https://arxiv.org/abs/2209.14345
作者：Jonah Anton,Harry Coppock,Pancham Shukla,Bjorn W. Schuller机构：Björn W. Schuller, GLAM, Imperial College London, UK & EIHW, University of Augsburg, Germany备注：15 pages (4 main text, rest references + appendices)摘要：Barlow Twins的自监督学习目标既不需要负样本也不需要非对称学习更新，实现了与计算机视觉中当前最先进技术水平相当的结果。因此，我们提出了一种新颖的自监督音频表示学习方法--音频Barlow Twins，它将Barlow Twins应用于音频领域。我们在大规模音频数据集AudioSet上进行了预训练，并在HEAR 2021挑战赛的18个任务上评估了学习到的表征的质量，获得的结果优于或等同于当前最先进的音频表征学习方法，例如区分自监督学习方法。代码，网址https://github.com/jonahanton/SSL_audio。摘要：The Barlow Twins self-supervised learning objective requires neither negative samples or asymmetric learning updates, achieving results on a par with the current state-of-the-art within Computer Vision. As such, we present Audio Barlow Twins, a novel self-supervised audio representation learning approach, adapting Barlow Twins to the audio domain. We pre-train on the large-scale audio dataset AudioSet, and evaluate the quality of the learnt representations on 18 tasks from the HEAR 2021 Challenge, achieving results which outperform, or otherwise are on a par with, the current state-of-the-art for instance discrimination self-supervised learning approaches to audio representation learning. Code at https://github.com/jonahanton/SSL_audio.
【5】 Text Independent Speaker Identification System for Access Control
标题：门禁系统中与文本无关的说话人识别系统
链接：https://arxiv.org/abs/2209.14335
机构：Dept. of Electrical and Electronic, Engineering, University of Ibadan, Ibadan, Nigeria摘要：即使是人类智能系统也不能提供100%的准确性来识别来自特定个体的语音。机器智能试图通过各种语音特征提取和语音建模技术来模拟人类在说话人识别问题中的行为。提出了一种基于Mel倒谱系数（MFCC）特征提取和k近邻（kNN）分类的文本无关说话人识别系统。获得的最大交叉验证准确度为60%。这将在以后的研究中加以改进。摘要：Even human intelligence system fails to offer 100% accuracy in identifying speeches from a specific individual. Machine intelligence is trying to mimic humans in speaker identification problems through various approaches to speech feature extraction and speech modeling techniques. This paper presents a text-independent speaker identification system that employs Mel Frequency Cepstral Coefficients (MFCC) for feature extraction and k-Nearest Neighbor (kNN) for classification. The maximum cross-validation accuracy obtained was 60%. This will be improved upon in subsequent research.
【1】 Text Independent Speaker Identification System for Access Control
标题：门禁系统中与文本无关的说话人识别系统
链接：https://arxiv.org/abs/2209.14335
* 与cs.SD语音【5】为同一篇
机构：Dept. of Electrical and Electronic, Engineering, University of Ibadan, Ibadan, Nigeria摘要：即使是人类智能系统也不能提供100%的准确性来识别来自特定个体的语音。机器智能试图通过各种语音特征提取和语音建模技术来模拟人类在说话人识别问题中的行为。提出了一种基于Mel倒谱系数（MFCC）特征提取和k近邻（kNN）分类的文本无关说话人识别系统。获得的最大交叉验证准确度为60%。这将在以后的研究中加以改进。摘要：Even human intelligence system fails to offer 100% accuracy in identifying speeches from a specific individual. Machine intelligence is trying to mimic humans in speaker identification problems through various approaches to speech feature extraction and speech modeling techniques. This paper presents a text-independent speaker identification system that employs Mel Frequency Cepstral Coefficients (MFCC) for feature extraction and k-Nearest Neighbor (kNN) for classification. The maximum cross-validation accuracy obtained was 60%. This will be improved upon in subsequent research.
【2】 ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognition
标题：ConvRNN-T：用于流语音识别的卷积增广递归神经网络换能器
链接：https://arxiv.org/abs/2209.14868
* 与cs.SD语音【1】为同一篇
作者：Martin Radfar,Rohit Barnwal,Rupak Vignesh Swaminathan,Feng-Ju Chang,Grant P. Strimel,Nathan Susanj,Athanasios Mouchtaris机构：Alexa Machine Learning, Amazon, USA备注：This paper was presented in Interspeech 2022摘要：递归神经网络传感器（RNN-T）是一种重要的流式端到端（E2 E）ASR技术。在RNN-T中，声编码器通常由LSTM的堆叠组成。最近，作为LSTM层的替代，Conformer架构被引入，其中RNN-T的编码器被修改的Transformer编码器代替，该编码器由前端和关注层之间的卷积层组成。本文提出了一种新的流式ASR模型--卷积增强递归神经网络转换器（Convronational Augmented Recurrent Neural Network Transducers，ConvRNN-T），它是在基于LSTM的RNN-T的基础上增加了一个由局部和全局上下文CNN编码器组成的卷积前端。ConvRNN-T利用因果1-D卷积层、挤压和激励、膨胀和残差块来向LSTM层提供全局和局部音频上下文表示。我们显示ConvRNN-T在Librispeech和内部数据上的性能优于RNN-T、Conformer和ContextNet。此外，与Conformer相比，ConvRNN-T的计算复杂度更低。ConvRNN-T的高精度和低占用空间使其成为设备上流式ASR技术的一个有前途的候选产品。摘要：The recurrent neural network transducer (RNN-T) is a prominent streaming end-to-end (E2E) ASR technology. In RNN-T, the acoustic encoder commonly consists of stacks of LSTMs. Very recently, as an alternative to LSTM layers, the Conformer architecture was introduced where the encoder of RNN-T is replaced with a modified Transformer encoder composed of convolutional layers at the frontend and between attention layers. In this paper, we introduce a new streaming ASR model, Convolutional Augmented Recurrent Neural Network Transducers (ConvRNN-T) in which we augment the LSTM-based RNN-T with a novel convolutional frontend consisting of local and global context CNN encoders. ConvRNN-T takes advantage of causal 1-D convolutional layers, squeeze-and-excitation, dilation, and residual blocks to provide both global and local audio context representation to LSTM layers. We show ConvRNN-T outperforms RNN-T, Conformer, and ContextNet on Librispeech and in-house data. In addition, ConvRNN-T offers less computational complexity compared to Conformer. ConvRNN-T's superior accuracy along with its low footprint make it a promising candidate for on-device streaming ASR technologies.
【3】 Classification of Vocal Bursts for ACII 2022 A-VB-Type Competition using Convolutional Network Networks and Deep Acoustic Embeddings
标题：基于卷积网络和深声嵌入的ACII2022A-VB型比赛发声群分类
链接：https://arxiv.org/abs/2209.14842
* 与cs.SD语音【2】为同一篇
作者：Muhammad Shehram Shah Syed,Zafi Sherhan Syed,Abbas Syed机构：Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro, Pakistan, Department of Telecommunication Engineering, Department of Electronic Engineering备注：Report for our submission to the ACII 2022 Affective Vocal Bursts (A-VB) Competition摘要：本报告简要描述了我们为ACII 2022情感爆发性发声（A-VB）竞赛的爆发性发声类型分类任务提出的解决方案。我们试验了两种方法，作为我们手头任务解决方案的一部分。第一种是基于Mel谱图训练的卷积神经网络，第二种是基于来自预先训练的wav 2 vec 2模型的深度声学嵌入的平均池。我们的最佳模型实现了测试分区的0.5190的未加权平均召回率（UAR），而机会水平UAR为0.1250，基线为0.4172。因此，比激发基线提高约20%。本文中报告的结果证明了我们提出的解决AV-B类型分类任务的方法的有效性。摘要：This report provides a brief description of our proposed solution for the Vocal Burst Type classification task of the ACII 2022 Affective Vocal Bursts (A-VB) Competition. We experimented with two approaches as part of our solution for the task at hand. The first of which is based on convolutional neural networks trained on Mel Spectrograms, and the second is based on average pooling of deep acoustic embeddings from a pretrained wav2vec2 model. Our best performing model achieves an unweighted average recall (UAR) of 0.5190 for the test partition, compared to the chance-level UAR of 0.1250 and a baseline of 0.4172. Thus, an improvement of around 20% over the challenge baseline. The results reported in this document demonstrate the efficacy of our proposed approaches to solve the AV-B Type Classification task.
【4】 The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling
标题：室群生成器：通过产生式建模获得无限高质量的MIR数据
链接：https://arxiv.org/abs/2209.14458
* 与cs.SD语音【3】为同一篇
作者：Yusong Wu,Josh Gardner,Ethan Manilow,Ian Simon,Curtis Hawthorne,Jesse Engel机构：Université de Montréal, Mila, University of Washington, Google Research, Brain Team摘要：数据是现代机器学习系统的生命线，包括音乐信息检索（MIR）中的数据。然而，长期以来，MIR一直被小数据集和不可靠的标签所困扰。在这项工作中，我们提出了打破这一瓶颈使用创成式建模。通过将音符生成模型（Coconet在Bach Chorales上训练）与室内乐合奏的结构化合成模型（MIDI-DDSP在URMP上训练）流水线化，我们展示了一个能够产生无限数量的具有丰富注释的真实合唱音乐的系统，包括混音、词干、MIDI、音符级演奏属性（断奏、颤音等），以及甚至细粒度的合成参数（音调、幅度等）。我们称这个系统为腔室集合生成器（CEG），并使用它从四个不同的腔室集合（CocoChorales）中生成大的合唱数据集。我们证明了使用我们的方法生成的数据改进了音乐转录和源分离的最先进模型，我们发布了系统和数据集，作为MIR社区未来工作的开源基础。摘要：Data is the lifeblood of modern machine learning systems, including for those in Music Information Retrieval (MIR). However, MIR has long been mired by small datasets and unreliable labels. In this work, we propose to break this bottleneck using generative modeling. By pipelining a generative model of notes (Coconet trained on Bach Chorales) with a structured synthesis model of chamber ensembles (MIDI-DDSP trained on URMP), we demonstrate a system capable of producing unlimited amounts of realistic chorale music with rich annotations including mixes, stems, MIDI, note-level performance attributes (staccato, vibrato, etc.), and even fine-grained synthesis parameters (pitch, amplitude, etc.). We call this system the Chamber Ensemble Generator (CEG), and use it to generate a large dataset of chorales from four different chamber ensembles (CocoChorales). We demonstrate that data generated using our approach improves state-of-the-art models for music transcription and source separation, and we release both the system and the dataset as an open-source foundation for future work in the MIR community.
机器翻译，仅供参考
每日语音速递