【1】 End-to-End Integration of Speech Recognition, Dereverberation, Beamforming, and Self-Supervised Learning Representation
标题：语音识别、混响消除、波束成形和自监督学习表示的端到端集成
链接：https://arxiv.org/abs/2210.10742
作者：Yoshiki Masuyama,Xuankai Chang,Samuele Cornell,Shinji Watanabe,Nobutaka Ono机构：Tokyo Metropolitan University, Japan, Carnegie Mellon University, USA, Italy备注：Accepted to IEEE SLT 2022摘要：自监督学习表示（SSLR）在自动语音识别（ASR）中表现出了显著的效果，主要是对干净语音的识别。最近的工作指出了在噪声环境中将SSLR与单通道语音增强相结合用于ASR的优势。本文通过对多通道输入的处理，进一步推进了这种集成。我们提出一种新颖的端对端架构，将去混响、波束成形、SSLR和ASR整合在单一神经网络中。我们的系统在CHiME-4 6通道轨道上实现了文献中报告的最佳性能，字错误率（WER）为1.77%。虽然基于WavLM的强SSLR本身展示了令人满意的结果，但与同时执行去混响和去噪的加权功率最小化无失真响应波束形成器的端到端集成显著改善了WER。在REVERB数据集上也验证了其有效性。摘要：Self-supervised learning representation (SSLR) has demonstrated its significant effectiveness in automatic speech recognition (ASR), mainly with clean speech. Recent work pointed out the strength of integrating SSLR with single-channel speech enhancement for ASR in noisy environments. This paper further advances this integration by dealing with multi-channel input. We propose a novel end-to-end architecture by integrating dereverberation, beamforming, SSLR, and ASR within a single neural network. Our system achieves the best performance reported in the literature on the CHiME-4 6-channel track with a word error rate (WER) of 1.77%. While the WavLM-based strong SSLR demonstrates promising results by itself, the end-to-end integration with the weighted power minimization distortionless response beamformer, which simultaneously performs dereverberation and denoising, improves WER significantly. Its effectiveness is also validated on the REVERB dataset.
【2】 Propagating Variational Model Uncertainty for Bioacoustic Call Label Smoothing
标题：生物声呼叫标签平滑的传播变分模型不确定性
链接：https://arxiv.org/abs/2210.10526
作者：Georgios Rizos,Jenna Lawson,Simon Mitchell,Pranay Shah,Xin Wen,Cristina Banks-Leite,Robert Ewers,Bjoern W. Schuller机构：and Bj¨orn W. Schuller, GLAM – Group on Language, Audio, & Music, Department of, Computing, Imperial College London, UK., Department of Life Sciences, Imperial College London, UK., DICE – Durrell Institute of Conservation and Ecology, University of Kent, UK.摘要：我们的重点是使用贝叶斯神经网络计算的预测不确定性信号来指导模型训练的同一任务中的学习，我们没有选择代价高昂的蒙特卡罗权重采样，我们以端到端的方式传播近似隐藏方差，通过带有注意和挤压-激励块的ResNet的变分贝叶斯自适应，以便识别对损失值计算贡献较小的数据样本。因此，我们提出了不确定性感知的、数据特定的标签平滑，其中平滑概率依赖于这种认识的不确定性。我们表明，通过在损失计算中显式地使用认知不确定性，变分模型导致了改进的预测和校准性能。这种核心的机器学习方法在野生动物叫声检测中得到了例证，通过动物自然栖息地中的被动声学监测设备进行音频记录，未来的目标是以可信的方式自动化大规模注释。摘要：We focus on using the predictive uncertainty signal calculated by Bayesian neural networks to guide learning in the self-same task the model is being trained on. Not opting for costly Monte Carlo sampling of weights, we propagate the approximate hidden variance in an end-to-end manner, throughout a variational Bayesian adaptation of a ResNet with attention and squeeze-and-excitation blocks, in order to identify data samples that should contribute less into the loss value calculation. We, thus, propose uncertainty-aware, data-specific label smoothing, where the smoothing probability is dependent on this epistemic uncertainty. We show that, through the explicit usage of the epistemic uncertainty in the loss calculation, the variational model is led to improved predictive and calibration performance. This core machine learning methodology is exemplified at wildlife call detection, from audio recordings made via passive acoustic monitoring equipment in the animals' natural habitats, with the future goal of automating large scale annotation in a trustworthy manner.
【3】 Audio Tampering Detection Based on Shallow and Deep Feature Representation Learning
标题：基于浅层和深层特征表示学习的音频篡改检测
链接：https://arxiv.org/abs/2210.10506
作者：Zhifeng Wang,Yao Yang,Chunyan Zeng,Shuai Kong,Shixiong Feng,Nan Zhao机构：Correspondence:, Department of Digital Media, Technology, Central China Normal, University, Luoyu Road , Hubei Key Laboratory for, High-efficiency Utilization of Solar, Energy and Operation Control of, Energy Storage System, Hubei, University of Technology, Nanli备注：Audio tampering detection, 21 pages, 4 figures摘要：数字音频篡改检测可用于验证数字音频的真实性。然而，大多数当前的方法使用标准电子网络频率（ENF）数据库来进行数字音频的ENF连续性的视觉比较分析，或者通过机器学习方法来执行用于分类的特征提取。ENF数据库通常难以获取，视觉方法的特征表示能力较弱，机器学习方法的特征信息丢失较多，导致检测准确率较低。为了充分利用ENF信息，提出了一种浅层和深层特征的融合方法，利用不同层次特征的互补性，更准确地描述原始数字音频篡改操作产生的不一致性变化。该方法在三个经典数据库上取得了97.03%的准确率：卡里奥卡1、卡里奥卡2和新西班牙语。另外，在新建立的数据库GAUDI-DI上，我们取得了88.31%的准确率。实验结果表明，该方法优于现有方法。摘要：Digital audio tampering detection can be used to verify the authenticity of digital audio. However, most current methods use standard electronic network frequency (ENF) databases for visual comparison analysis of ENF continuity of digital audio or perform feature extraction for classification by machine learning methods. ENF databases are usually tricky to obtain, visual methods have weak feature representation, and machine learning methods have more information loss in features, resulting in low detection accuracy. This paper proposes a fusion method of shallow and deep features to fully use ENF information by exploiting the complementary nature of features at different levels to more accurately describe the changes in inconsistency produced by tampering operations to raw digital audio. The method achieves 97.03% accuracy on three classic databases: Carioca 1, Carioca 2, and New Spanish. In addition, we have achieved an accuracy of 88.31% on the newly constructed database GAUDI-DI. Experimental results show that the proposed method is superior to the state-of-the-art method.
【4】 Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation
标题：MuseForm：音乐生成的细粒度和粗粒度关注的转换器
链接：https://arxiv.org/abs/2210.10349
作者：Botao Yu,Peiling Lu,Rui Wang,Wei Hu,Xu Tan,Wei Ye,Shikun Zhang,Tao Qin,Tie-Yan Liu机构：†State Key Laboratory for Novel Software Technology, Nanjing University, China, ‡Microsoft Research Asia, §National Engineering Research Center for Software Engineering, Peking University, China备注：Accepted by the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022)摘要：符号音乐生成的目的是自动生成乐谱。最近的趋势是在音乐生成中使用Transformer或其变体，然而，这是次优的，因为全部注意力不能有效地对通常较长的音乐序列（例如，超过10，000个记号），并且现有模型在生成音乐重复结构方面存在缺点。本文提出了一种新的基于细粒度和粗粒度注意力的音乐生成Transformer Museformer。具体地，利用细粒度关注，特定小节的表征直接关注与音乐结构最相关的小节的所有表征（例如，通过相似性统计选择的先前的第一、第二、第四和第八条）;通过粗粒度的关注，令牌只关注其他条的汇总，而不是它们的每个令牌，从而降低了计算成本。其优点有两方面。首先，它可以通过细粒度的注意力捕捉与音乐结构相关的相关性，并通过粗粒度的注意力捕捉其他上下文信息。第二，它是高效的，与全注意力对应物相比，它可以建模超过3倍长的音乐序列。客观和主观实验结果表明，该算法能够生成高质量、结构较好的长音乐序列。摘要：Symbolic music generation aims to generate music scores automatically. A recent trend is to use Transformer or its variants in music generation, which is, however, suboptimal, because the full attention cannot efficiently model the typically long music sequences (e.g., over 10,000 tokens), and the existing models have shortcomings in generating musical repetition structures. In this paper, we propose Museformer, a Transformer with a novel fine- and coarse-grained attention for music generation. Specifically, with the fine-grained attention, a token of a specific bar directly attends to all the tokens of the bars that are most relevant to music structures (e.g., the previous 1st, 2nd, 4th and 8th bars, selected via similarity statistics); with the coarse-grained attention, a token only attends to the summarization of the other bars rather than each token of them so as to reduce the computational cost. The advantages are two-fold. First, it can capture both music structure-related correlations via the fine-grained attention, and other contextual information via the coarse-grained attention. Second, it is efficient and can model over 3X longer music sequences compared to its full-attention counterpart. Both objective and subjective experimental results demonstrate its ability to generate long music sequences with high quality and better structures.
【5】 Two-stage training method for Japanese electrolaryngeal speech enhancement based on sequence-to-sequence voice conversion
标题：一种基于序列到序列语音转换的日语喉电语音增强两阶段训练方法
链接：https://arxiv.org/abs/2210.10314
作者：Ding Ma,Lester Phillip Violeta,Kazuhiro Kobayashi,Tomoki Toda机构：Nagoya University, Japan摘要：序列到序列（seq2seq）语音转换（VC）模型与传统VC模型相比在将电喉（EL）语音转换为正常语音（EL2SP）方面具有更大的潜力。然而，基于seq2seq VC的EL2SP算法需要足够大的并行数据量进行模型训练，当训练数据量不足时，EL2SP算法的性能会显著下降。针对这一问题，提出了一种新的两阶段策略，在并行数据量较少的情况下，对基于seq2seq VC的EL2SP进行性能优化.与以往研究中使用高质量数据增强不同，我们首先将大量不完美的EL和正常语音的合成并行数据与原始数据集结合到VC训练中。然后，仅使用原始并行数据集进行第二阶段训练。实验结果表明，该方法逐步提高了基于seq2seq VC的EL2SP算法的性能。摘要：Sequence-to-sequence (seq2seq) voice conversion (VC) models have greater potential in converting electrolaryngeal (EL) speech to normal speech (EL2SP) compared to conventional VC models. However, EL2SP based on seq2seq VC requires a sufficiently large amount of parallel data for the model training and it suffers from significant performance degradation when the amount of training data is insufficient. To address this issue, we suggest a novel, two-stage strategy to optimize the performance on EL2SP based on seq2seq VC when a small amount of the parallel dataset is available. In contrast to utilizing high-quality data augmentations in previous studies, we first combine a large amount of imperfect synthetic parallel data of EL and normal speech, with the original dataset into VC training. Then, a second stage training is conducted with the original parallel dataset only. The results show that the proposed method progressively improves the performance of EL2SP based on seq2seq VC.
【6】 A Data-Driven Investigation of Noise-Adaptive Utterance Generation with Linguistic Modification
标题：基于数据驱动的语言修改噪声自适应语音生成研究
链接：https://arxiv.org/abs/2210.10252
作者：Anupama Chingacham,Vera Demberg,Dietrich Klakow机构：Saarland Informatics Campus, Saarland University, Germany, ©Copyright , IEEE. Published in the , IEEE Spoken Language Technology Workshop (SLT) (SLT ,)摘要：在嘈杂的环境中，人类可能难以理解语音。口头对话系统可以帮助增强其输出的可理解性，这或者通过修改语音合成（例如，模仿伦巴第语）或通过优化语言生成。我们在这里关注的是第二种方法，通过这种方法，一个预期的信息是用在特定的嘈杂环境中更容易理解的单词来实现的。通过进行语音感知实验，我们创建了一个由900个释义的数据集，这些释义由听力正常的英语母语者感知。我们发现，仔细选择释义可以在SNR为-5dB时提高可懂度33%。我们对数据的分析表明，释义之间的可懂度差异主要是由噪声鲁棒的声学线索驱动的。在此基础上，提出了一种可懂度感知的释义排序模型，在信噪比为-5dB时，该模型的性能相对于基准模型提高了31.37%。摘要：In noisy environments, speech can be hard to understand for humans. Spoken dialog systems can help to enhance the intelligibility of their output, either by modifying the speech synthesis (e.g., imitate Lombard speech) or by optimizing the language generation. We here focus on the second type of approach, by which an intended message is realized with words that are more intelligible in a specific noisy environment. By conducting a speech perception experiment, we created a dataset of 900 paraphrases in babble noise, perceived by native English speakers with normal hearing. We find that careful selection of paraphrases can improve intelligibility by 33% at SNR -5 dB. Our analysis of the data shows that the intelligibility differences between paraphrases are mainly driven by noise-robust acoustic cues. Furthermore, we propose an intelligibility-aware paraphrase ranking model, which outperforms baseline models with a relative improvement of 31.37% at SNR -5 dB.
【7】 Speaker- and Age-Invariant Training for Child Acoustic Modeling Using Adversarial Multi-Task Learning
标题：使用对抗性多任务学习的儿童声学建模的说话人和年龄不变的训练
链接：https://arxiv.org/abs/2210.10231
作者：Mostafa Shahin,Beena Ahmed,Julien Epps机构：School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, Australia备注：Submitted to ICASSP2023摘要：儿童语音声学建模的主要挑战之一是儿童的发音器官随着他们的成长而发生的快速变化、他们不同的生长速率以及随后在同一年龄组中的高可变性。这些高的声学变化以及儿童语音语料库的缺乏阻碍了可靠的儿童语音识别系统的开发。本文提出了一种基于对抗式多任务学习的说话人和年龄不变的训练方法。该系统由一个生成器共享网络组成，该网络学习生成说话人和年龄不变特征，并连接到三个区分网络，分别用于音素、年龄和说话人。该发生器网络被训练成以对抗性多任务学习方式最小化音素辨别损失并最大化说话人和年龄辨别损失。发生器网络是一个时间延迟神经网络（TDNN）结构，而三个鉴别器是前馈网络。将该系统应用于OGI语音库，ASR的WER降低了13%。摘要：One of the major challenges in acoustic modelling of child speech is the rapid changes that occur in the children's articulators as they grow up, their differing growth rates and the subsequent high variability in the same age group. These high acoustic variations along with the scarcity of child speech corpora have impeded the development of a reliable speech recognition system for children. In this paper, a speaker- and age-invariant training approach based on adversarial multi-task learning is proposed. The system consists of one generator shared network that learns to generate speaker- and age-invariant features connected to three discrimination networks, for phoneme, age, and speaker. The generator network is trained to minimize the phoneme-discrimination loss and maximize the speaker- and age-discrimination losses in an adversarial multi-task learning fashion. The generator network is a Time Delay Neural Network (TDNN) architecture while the three discriminators are feed-forward networks. The system was applied to the OGI speech corpora and achieved a 13% reduction in the WER of the ASR.
【8】 BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds
标题：鸟声去噪：鸟鸣的深度视觉去噪
链接：https://arxiv.org/abs/2210.10196
作者：Youshan Zhang,Jialu Li机构：Yeshiva University, NYC, NY, Cornell University, Ithaca, NY摘要：音频去噪已经使用传统和基于深度学习的方法进行了几十年的探索。然而，这些方法仍然局限于手动添加的人工噪声或较低的去噪音频质量。为了克服这些挑战，我们收集了大规模自然噪声鸟声数据集。我们首次将音频去噪问题转化为图像分割问题，提出了一种深度视觉音频去噪（DVAD）模型。我们开发了一个音频ImageMask工具，并建议使用一个Few-Shot泛化策略来标记这些图像。大量实验结果表明，该模型具有较好的性能。我们还表明，我们的方法可以很容易地推广到语音去噪，音频分离，音频增强，和噪声估计。摘要：Audio denoising has been explored for decades using both traditional and deep learning-based methods. However, these methods are still limited to either manually added artificial noise or lower denoised audio quality. To overcome these challenges, we collect a large-scale natural noise bird sound dataset. We are the first to transfer the audio denoising problem into an image segmentation problem and propose a deep visual audio denoising (DVAD) model. With a total of 14,120 audio images, we develop an audio ImageMask tool and propose to use a few-shot generalization strategy to label these images. Extensive experimental results demonstrate that the proposed model achieves state-of-the-art performance. We also show that our method can be easily generalized to speech denoising, audio separation, audio enhancement, and noise estimation.
【9】 Simple and Effective Unsupervised Speech Translation
标题：简单有效的无监督语音翻译
链接：https://arxiv.org/abs/2210.10191
作者：Changhan Wang,Hirofumi Inaguma,Peng-Jen Chen,Ilia Kulikov,Yun Tang,Wei-Ning Hsu,Michael Auli,Juan Pino机构：Meta - Fundamental AI Research (FAIR)摘要：对于大多数语言来说，用于训练语音任务模型的标记数据量是有限的，然而，对于需要覆盖两种不同语言的标记数据的语音翻译来说，数据稀缺性加剧。为了解决这个问题，我们研究了一种简单有效的方法，利用无监督语音识别、机器翻译和语音合成的最新进展，在没有标记数据的情况下构建语音翻译系统，或者在流水线方法中，或者生成用于训练端到端语音翻译模型的伪标记。此外，我们提出了一种用于预训练语音模型的无监督域自适应技术，该技术提高了下游无监督语音识别的性能，特别是在低资源设置下。实验表明，在Libri-Trans基准测试中，无监督语音到文本翻译的性能比以前的无监督技术水平高3.2 BLEU，在CoVoST 2上，我们的最佳系统在五个X-En方向上的平均性能比仅两年前的最佳有监督端到端模型（无预训练）高5.0 BLEU。我们还报告了在Must-C和CVSS基准测试中的竞争结果。摘要：The amount of labeled data to train models for speech tasks is limited for most languages, however, the data scarcity is exacerbated for speech translation which requires labeled data covering two different languages. To address this issue, we study a simple and effective approach to build speech translation systems without labeled data by leveraging recent advances in unsupervised speech recognition, machine translation and speech synthesis, either in a pipeline approach, or to generate pseudo-labels for training end-to-end speech translation models. Furthermore, we present an unsupervised domain adaptation technique for pre-trained speech models which improves the performance of downstream unsupervised speech recognition, especially for low-resource settings. Experiments show that unsupervised speech-to-text translation outperforms the previous unsupervised state of the art by 3.2 BLEU on the Libri-Trans benchmark, on CoVoST 2, our best systems outperform the best supervised end-to-end models (without pre-training) from only two years ago by an average of 5.0 BLEU over five X-En directions. We also report competitive results on MuST-C and CVSS benchmarks.
【10】 Spoofed training data for speech spoofing countermeasure can be efficiently created using neural vocoders
标题：可以使用神经声码器有效地创建用于语音欺骗对抗的欺骗训练数据
链接：https://arxiv.org/abs/2210.10570
作者：Xin Wang,Junichi Yamagishi机构：National Institute of Informatics, Japan备注：ICASSP 2023 submission摘要：用于语音欺骗对抗的好的训练集需要不同的TTS和VC欺骗攻击，但是为目标说话者生成TTS和VC欺骗试验可能在技术上要求很高。本研究不使用成熟的TTS和VC系统，而是使用基于神经网络的声码器对真实话语进行拷贝合成。输出数据可用作欺骗数据。为了更好地利用真实和欺骗数据对，本研究引入了一种可插入标准训练准则的对比特征损失。基于ASVspoof 2019逻辑访问训练集的真实试验，本研究实证地比较了使用几个神经非自回归声码器以所提出的方式创建的几个训练集。多个测试集的结果表明了良好的实践，如使用来自目标域的真实数据微调神经声码器。实验结果也证明了对比特征损失的有效性。通过结合最佳实践，经过培训的CM实现了整体竞争绩效。其在ASVspoof 2021隐藏子集上的EER也优于前1名的挑战提交。摘要：A good training set for speech spoofing countermeasures requires diverse TTS and VC spoofing attacks, but generating TTS and VC spoofed trials for a target speaker may be technically demanding. Instead of using full-fledged TTS and VC systems, this study uses neural-network-based vocoders to do copy-synthesis on bona fide utterances. The output data can be used as spoofed data. To make better use of pairs of bona fide and spoofed data, this study introduces a contrastive feature loss that can be plugged into the standard training criterion. On the basis of the bona fide trials from the ASVspoof 2019 logical access training set, this study empirically compared a few training sets created in the proposed manner using a few neural non-autoregressive vocoders. Results on multiple test sets suggest good practices such as fine-tuning neural vocoders using bona fide data from the target domain. The results also demonstrated the effectiveness of the contrastive feature loss. Combining the best practices, the trained CM achieved overall competitive performance. Its EERs on the ASVspoof 2021 hidden subsets also outperformed the top-1 challenge submissions.
【11】 Deep Learning Based Two-dimensional Speaker Localization With Large Ad-hoc Microphone Arrays
标题：基于深度学习的大型自组织麦克风阵列二维说话人定位
链接：https://arxiv.org/abs/2210.10265
作者：Shupei Liu,Yijun Gong,Xiao-Lei Zhang,Xuelong Li摘要：基于深度学习的说话人定位方法在混响环境中显示出了其优越性。然而，它主要集中在说话人定位的波达方向（DOA）估计子任务上，其中仅获得DOA而不是二维（2D）坐标。为了获得任意位置的多个说话人的二维坐标，提出了一种基于深度学习的大规模自组织麦克风阵列二维说话人定位方法。具体地说，将卷积神经网络应用于每个节点以获得语音源的到达方向（DOA）估计。然后，三角测量和聚类方法结合节点的DOA估计来估计语音源的2D位置。为了进一步提高估计精度，提出了一种基于softmax的节点选择算法。在大规模自组织麦克风阵列上的实验结果表明，该方法在仿真和真实环境中均取得了比传统方法更好的性能。基于softmax的节点选择进一步提高了性能。摘要：Deep learning based speaker localization has shown its advantage in reverberant scenarios. However, it mostly focuses on the direction-of-arrival (DOA) estimation subtask of speaker localization, where the DOA instead of the 2-dimensional (2D) coordinates is obtained only. To obtain the 2D coordinates of multiple speakers with random positions, this paper proposes a deep-learning-based 2D speaker localization method with large ad-hoc microphone arrays, where an ad-hoc microphone array is a set of randomly-distributed microphone nodes with each node set to a traditional microphone array, e.g. a linear array. Specifically, a convolutional neural network is applied to each node to get the direction-of-arrival (DOA) estimation of speech sources. Then, a triangulation and clustering method integrates the DOA estimations of the nodes for estimating the 2D positions of the speech sources. To further improve the estimation accuracy, we propose a softmax-based node selection algorithm. Experimental results with large-scale ad-hoc microphone arrays show that the proposed method achieves significantly better performance than conventional methods in both simulated and real-world environments. The softmax-based node selection further improves the performance.
【12】 Multi-Source Transformer Architectures for Audiovisual Scene Classification
标题：用于视听场景分类的多源Transformer体系结构
链接：https://arxiv.org/abs/2210.10212
作者：Wim Boes,Hugo Van hamme备注：Technical report of submission to DCASE 2021 Challenge Task 1B摘要：在本技术报告中，详细描述了我们为DCASE 2021挑战的子任务1B提交的关于视听场景分类的系统。它们本质上是多源Transformers，采用听觉和视觉特征的组合来进行预测。利用宏观平均的多类交叉熵和准确性度量对这些模型进行评估。 在宏观平均多类交叉熵方面，我们的最佳模型在验证数据上获得了0.620的得分。这比基准系统的性能（0.658）稍好。 在准确性方面，我们的最佳模型在验证数据上获得了77.1%的得分，与基线系统获得的性能（77.0%）大致相同.摘要：In this technical report, the systems we submitted for subtask 1B of the DCASE 2021 challenge, regarding audiovisual scene classification, are described in detail. They are essentially multi-source transformers employing a combination of auditory and visual features to make predictions. These models are evaluated utilizing the macro-averaged multi-class cross-entropy and accuracy metrics. In terms of the macro-averaged multi-class cross-entropy, our best model achieved a score of 0.620 on the validation data. This is slightly better than the performance of the baseline system (0.658). With regard to the accuracy measure, our best model achieved a score of 77.1\% on the validation data, which is about the same as the performance obtained by the baseline system (77.0\%).
【13】 Optimizing Temporal Resolution Of Convolutional Recurrent Neural Networks For Sound Event Detection
标题：用于声事件检测的卷积递归神经网络时间分辨率优化
链接：https://arxiv.org/abs/2210.10208
作者：Wim Boes,Hugo Van hamme备注：Technical report of submission to DCASE 2021 Challenge Task 4摘要：在本技术报告中，我们详细描述了我们为DCASE 2021挑战的子任务4（关于声音事件检测）提交的系统。这些模型与为该问题提供的基线密切相关，因为它们本质上是在平均教师设置中训练的卷积递归神经网络，以处理所提供数据的异构注释。然而，对预测的时间分辨率进行了调整，以处理这样的事实，即这些系统是使用两个基于相交的度量来评估的，这两个度量涉及在时间定位方面的不同需求。这是通过优化合并操作实现的。 对于第一个定义的评估场景，对时间定位精度提出了相对严格的要求，我们的最佳模型在验证数据上获得了0.3609的PSDS分数。这仅略好于基线系统获得的性能（0.342）：基线网络中的池化量已经证明是最佳的，因此没有进行实质性的改变，这解释了这一结果。 对于第二种评估场景，对定位精度施加相对宽松的限制，我们性能最好的系统在验证数据上获得了0.7312的PSDS分数。这明显好于基线模型获得的性能（0.527），这可以有效地归因于应用于网络的池化操作的改变。摘要：In this technical report, the systems we submitted for subtask 4 of the DCASE 2021 challenge, regarding sound event detection, are described in detail. These models are closely related to the baseline provided for this problem, as they are essentially convolutional recurrent neural networks trained in a mean teacher setting to deal with the heterogeneous annotation of the supplied data. However, the time resolution of the predictions was adapted to deal with the fact that these systems are evaluated using two intersection-based metrics involving different needs in terms of temporal localization. This was done by optimizing the pooling operations. For the first of the defined evaluation scenarios, imposing relatively strict requirements on the temporal localization accuracy, our best model achieved a PSDS score of 0.3609 on the validation data. This is only marginally better than the performance obtained by the baseline system (0.342): The amount of pooling in the baseline network already turned out to be optimal, and thus, no substantial changes were made, explaining this result. For the second evaluation scenario, imposing relatively lax restrictions on the localization accuracy, our best-performing system achieved a PSDS score of 0.7312 on the validation data. This is significantly better than the performance obtained by the baseline model (0.527), which can effectively be attributed to the changes that were applied to the pooling operations of the network.
【1】 Spoofed training data for speech spoofing countermeasure can be efficiently created using neural vocoders
标题：可以使用神经声码器有效地创建用于语音欺骗对抗的欺骗训练数据
链接：https://arxiv.org/abs/2210.10570
* 与cs.SD语音【10】为同一篇
作者：Xin Wang,Junichi Yamagishi机构：National Institute of Informatics, Japan备注：ICASSP 2023 submission摘要：用于语音欺骗对抗的好的训练集需要不同的TTS和VC欺骗攻击，但是为目标说话者生成TTS和VC欺骗试验可能在技术上要求很高。本研究不使用成熟的TTS和VC系统，而是使用基于神经网络的声码器对真实话语进行拷贝合成。输出数据可用作欺骗数据。为了更好地利用真实和欺骗数据对，本研究引入了一种可插入标准训练准则的对比特征损失。基于ASVspoof 2019逻辑访问训练集的真实试验，本研究实证地比较了使用几个神经非自回归声码器以所提出的方式创建的几个训练集。多个测试集的结果表明了良好的实践，如使用来自目标域的真实数据微调神经声码器。实验结果也证明了对比特征损失的有效性。通过结合最佳实践，经过培训的CM实现了整体竞争绩效。其在ASVspoof 2021隐藏子集上的EER也优于前1名的挑战提交。摘要：A good training set for speech spoofing countermeasures requires diverse TTS and VC spoofing attacks, but generating TTS and VC spoofed trials for a target speaker may be technically demanding. Instead of using full-fledged TTS and VC systems, this study uses neural-network-based vocoders to do copy-synthesis on bona fide utterances. The output data can be used as spoofed data. To make better use of pairs of bona fide and spoofed data, this study introduces a contrastive feature loss that can be plugged into the standard training criterion. On the basis of the bona fide trials from the ASVspoof 2019 logical access training set, this study empirically compared a few training sets created in the proposed manner using a few neural non-autoregressive vocoders. Results on multiple test sets suggest good practices such as fine-tuning neural vocoders using bona fide data from the target domain. The results also demonstrated the effectiveness of the contrastive feature loss. Combining the best practices, the trained CM achieved overall competitive performance. Its EERs on the ASVspoof 2021 hidden subsets also outperformed the top-1 challenge submissions.
【2】 Deep Learning Based Two-dimensional Speaker Localization With Large Ad-hoc Microphone Arrays
标题：基于深度学习的大型自组织麦克风阵列二维说话人定位
链接：https://arxiv.org/abs/2210.10265
* 与cs.SD语音【11】为同一篇
作者：Shupei Liu,Yijun Gong,Xiao-Lei Zhang,Xuelong Li摘要：基于深度学习的说话人定位方法在混响环境中显示出了其优越性。然而，它主要集中在说话人定位的波达方向（DOA）估计子任务上，其中仅获得DOA而不是二维（2D）坐标。为了获得任意位置的多个说话人的二维坐标，提出了一种基于深度学习的大规模自组织麦克风阵列二维说话人定位方法。具体地说，将卷积神经网络应用于每个节点以获得语音源的到达方向（DOA）估计。然后，三角测量和聚类方法结合节点的DOA估计来估计语音源的2D位置。为了进一步提高估计精度，提出了一种基于softmax的节点选择算法。在大规模自组织麦克风阵列上的实验结果表明，该方法在仿真和真实环境中均取得了比传统方法更好的性能。基于softmax的节点选择进一步提高了性能。摘要：Deep learning based speaker localization has shown its advantage in reverberant scenarios. However, it mostly focuses on the direction-of-arrival (DOA) estimation subtask of speaker localization, where the DOA instead of the 2-dimensional (2D) coordinates is obtained only. To obtain the 2D coordinates of multiple speakers with random positions, this paper proposes a deep-learning-based 2D speaker localization method with large ad-hoc microphone arrays, where an ad-hoc microphone array is a set of randomly-distributed microphone nodes with each node set to a traditional microphone array, e.g. a linear array. Specifically, a convolutional neural network is applied to each node to get the direction-of-arrival (DOA) estimation of speech sources. Then, a triangulation and clustering method integrates the DOA estimations of the nodes for estimating the 2D positions of the speech sources. To further improve the estimation accuracy, we propose a softmax-based node selection algorithm. Experimental results with large-scale ad-hoc microphone arrays show that the proposed method achieves significantly better performance than conventional methods in both simulated and real-world environments. The softmax-based node selection further improves the performance.
【3】 Multi-Source Transformer Architectures for Audiovisual Scene Classification
标题：用于视听场景分类的多源Transformer体系结构
链接：https://arxiv.org/abs/2210.10212
* 与cs.SD语音【12】为同一篇
作者：Wim Boes,Hugo Van hamme备注：Technical report of submission to DCASE 2021 Challenge Task 1B摘要：在本技术报告中，详细描述了我们为DCASE 2021挑战的子任务1B提交的关于视听场景分类的系统。它们本质上是多源Transformers，采用听觉和视觉特征的组合来进行预测。利用宏观平均的多类交叉熵和准确性度量对这些模型进行评估。 在宏观平均多类交叉熵方面，我们的最佳模型在验证数据上获得了0.620的得分。这比基准系统的性能（0.658）稍好。 在准确性方面，我们的最佳模型在验证数据上获得了77.1%的得分，与基线系统获得的性能（77.0%）大致相同.摘要：In this technical report, the systems we submitted for subtask 1B of the DCASE 2021 challenge, regarding audiovisual scene classification, are described in detail. They are essentially multi-source transformers employing a combination of auditory and visual features to make predictions. These models are evaluated utilizing the macro-averaged multi-class cross-entropy and accuracy metrics. In terms of the macro-averaged multi-class cross-entropy, our best model achieved a score of 0.620 on the validation data. This is slightly better than the performance of the baseline system (0.658). With regard to the accuracy measure, our best model achieved a score of 77.1\% on the validation data, which is about the same as the performance obtained by the baseline system (77.0\%).
【4】 Optimizing Temporal Resolution Of Convolutional Recurrent Neural Networks For Sound Event Detection
标题：用于声事件检测的卷积递归神经网络时间分辨率优化
链接：https://arxiv.org/abs/2210.10208
* 与cs.SD语音【13】为同一篇
作者：Wim Boes,Hugo Van hamme备注：Technical report of submission to DCASE 2021 Challenge Task 4摘要：在本技术报告中，我们详细描述了我们为DCASE 2021挑战的子任务4（关于声音事件检测）提交的系统。这些模型与为该问题提供的基线密切相关，因为它们本质上是在平均教师设置中训练的卷积递归神经网络，以处理所提供数据的异构注释。然而，对预测的时间分辨率进行了调整，以处理这样的事实，即这些系统是使用两个基于相交的度量来评估的，这两个度量涉及在时间定位方面的不同需求。这是通过优化合并操作实现的。 对于第一个定义的评估场景，对时间定位精度提出了相对严格的要求，我们的最佳模型在验证数据上获得了0.3609的PSDS分数。这仅略好于基线系统获得的性能（0.342）：基线网络中的池化量已经证明是最佳的，因此没有进行实质性的改变，这解释了这一结果。 对于第二种评估场景，对定位精度施加相对宽松的限制，我们性能最好的系统在验证数据上获得了0.7312的PSDS分数。这明显好于基线模型获得的性能（0.527），这可以有效地归因于应用于网络的池化操作的改变。摘要：In this technical report, the systems we submitted for subtask 4 of the DCASE 2021 challenge, regarding sound event detection, are described in detail. These models are closely related to the baseline provided for this problem, as they are essentially convolutional recurrent neural networks trained in a mean teacher setting to deal with the heterogeneous annotation of the supplied data. However, the time resolution of the predictions was adapted to deal with the fact that these systems are evaluated using two intersection-based metrics involving different needs in terms of temporal localization. This was done by optimizing the pooling operations. For the first of the defined evaluation scenarios, imposing relatively strict requirements on the temporal localization accuracy, our best model achieved a PSDS score of 0.3609 on the validation data. This is only marginally better than the performance obtained by the baseline system (0.342): The amount of pooling in the baseline network already turned out to be optimal, and thus, no substantial changes were made, explaining this result. For the second evaluation scenario, imposing relatively lax restrictions on the localization accuracy, our best-performing system achieved a PSDS score of 0.7312 on the validation data. This is significantly better than the performance obtained by the baseline model (0.527), which can effectively be attributed to the changes that were applied to the pooling operations of the network.
【5】 End-to-End Integration of Speech Recognition, Dereverberation, Beamforming, and Self-Supervised Learning Representation
标题：语音识别、混响消除、波束成形和自监督学习表示的端到端集成
链接：https://arxiv.org/abs/2210.10742
* 与cs.SD语音【1】为同一篇
作者：Yoshiki Masuyama,Xuankai Chang,Samuele Cornell,Shinji Watanabe,Nobutaka Ono机构：Tokyo Metropolitan University, Japan, Carnegie Mellon University, USA, Italy备注：Accepted to IEEE SLT 2022摘要：自监督学习表示（SSLR）在自动语音识别（ASR）中表现出了显著的效果，主要是对干净语音的识别。最近的工作指出了在噪声环境中将SSLR与单通道语音增强相结合用于ASR的优势。本文通过对多通道输入的处理，进一步推进了这种集成。我们提出一种新颖的端对端架构，将去混响、波束成形、SSLR和ASR整合在单一神经网络中。我们的系统在CHiME-4 6通道轨道上实现了文献中报告的最佳性能，字错误率（WER）为1.77%。虽然基于WavLM的强SSLR本身展示了令人满意的结果，但与同时执行去混响和去噪的加权功率最小化无失真响应波束形成器的端到端集成显著改善了WER。在REVERB数据集上也验证了其有效性。摘要：Self-supervised learning representation (SSLR) has demonstrated its significant effectiveness in automatic speech recognition (ASR), mainly with clean speech. Recent work pointed out the strength of integrating SSLR with single-channel speech enhancement for ASR in noisy environments. This paper further advances this integration by dealing with multi-channel input. We propose a novel end-to-end architecture by integrating dereverberation, beamforming, SSLR, and ASR within a single neural network. Our system achieves the best performance reported in the literature on the CHiME-4 6-channel track with a word error rate (WER) of 1.77%. While the WavLM-based strong SSLR demonstrates promising results by itself, the end-to-end integration with the weighted power minimization distortionless response beamformer, which simultaneously performs dereverberation and denoising, improves WER significantly. Its effectiveness is also validated on the REVERB dataset.
【6】 Propagating Variational Model Uncertainty for Bioacoustic Call Label Smoothing
标题：生物声呼叫标签平滑的传播变分模型不确定性
链接：https://arxiv.org/abs/2210.10526
* 与cs.SD语音【2】为同一篇
作者：Georgios Rizos,Jenna Lawson,Simon Mitchell,Pranay Shah,Xin Wen,Cristina Banks-Leite,Robert Ewers,Bjoern W. Schuller机构：and Bj¨orn W. Schuller, GLAM – Group on Language, Audio, & Music, Department of, Computing, Imperial College London, UK., Department of Life Sciences, Imperial College London, UK., DICE – Durrell Institute of Conservation and Ecology, University of Kent, UK.摘要：我们的重点是使用贝叶斯神经网络计算的预测不确定性信号来指导模型训练的同一任务中的学习，我们没有选择代价高昂的蒙特卡罗权重采样，我们以端到端的方式传播近似隐藏方差，通过带有注意和挤压-激励块的ResNet的变分贝叶斯自适应，以便识别对损失值计算贡献较小的数据样本。因此，我们提出了不确定性感知的、数据特定的标签平滑，其中平滑概率依赖于这种认识的不确定性。我们表明，通过在损失计算中显式地使用认知不确定性，变分模型导致了改进的预测和校准性能。这种核心的机器学习方法在野生动物叫声检测中得到了例证，通过动物自然栖息地中的被动声学监测设备进行音频记录，未来的目标是以可信的方式自动化大规模注释。摘要：We focus on using the predictive uncertainty signal calculated by Bayesian neural networks to guide learning in the self-same task the model is being trained on. Not opting for costly Monte Carlo sampling of weights, we propagate the approximate hidden variance in an end-to-end manner, throughout a variational Bayesian adaptation of a ResNet with attention and squeeze-and-excitation blocks, in order to identify data samples that should contribute less into the loss value calculation. We, thus, propose uncertainty-aware, data-specific label smoothing, where the smoothing probability is dependent on this epistemic uncertainty. We show that, through the explicit usage of the epistemic uncertainty in the loss calculation, the variational model is led to improved predictive and calibration performance. This core machine learning methodology is exemplified at wildlife call detection, from audio recordings made via passive acoustic monitoring equipment in the animals' natural habitats, with the future goal of automating large scale annotation in a trustworthy manner.
【7】 Audio Tampering Detection Based on Shallow and Deep Feature Representation Learning
标题：基于浅层和深层特征表示学习的音频篡改检测
链接：https://arxiv.org/abs/2210.10506
* 与cs.SD语音【3】为同一篇
作者：Zhifeng Wang,Yao Yang,Chunyan Zeng,Shuai Kong,Shixiong Feng,Nan Zhao机构：Correspondence:, Department of Digital Media, Technology, Central China Normal, University, Luoyu Road , Hubei Key Laboratory for, High-efficiency Utilization of Solar, Energy and Operation Control of, Energy Storage System, Hubei, University of Technology, Nanli备注：Audio tampering detection, 21 pages, 4 figures摘要：数字音频篡改检测可用于验证数字音频的真实性。然而，大多数当前的方法使用标准电子网络频率（ENF）数据库来进行数字音频的ENF连续性的视觉比较分析，或者通过机器学习方法来执行用于分类的特征提取。ENF数据库通常难以获取，视觉方法的特征表示能力较弱，机器学习方法的特征信息丢失较多，导致检测准确率较低。为了充分利用ENF信息，提出了一种浅层和深层特征的融合方法，利用不同层次特征的互补性，更准确地描述原始数字音频篡改操作产生的不一致性变化。该方法在三个经典数据库上取得了97.03%的准确率：卡里奥卡1、卡里奥卡2和新西班牙语。另外，在新建立的数据库GAUDI-DI上，我们取得了88.31%的准确率。实验结果表明，该方法优于现有方法。摘要：Digital audio tampering detection can be used to verify the authenticity of digital audio. However, most current methods use standard electronic network frequency (ENF) databases for visual comparison analysis of ENF continuity of digital audio or perform feature extraction for classification by machine learning methods. ENF databases are usually tricky to obtain, visual methods have weak feature representation, and machine learning methods have more information loss in features, resulting in low detection accuracy. This paper proposes a fusion method of shallow and deep features to fully use ENF information by exploiting the complementary nature of features at different levels to more accurately describe the changes in inconsistency produced by tampering operations to raw digital audio. The method achieves 97.03% accuracy on three classic databases: Carioca 1, Carioca 2, and New Spanish. In addition, we have achieved an accuracy of 88.31% on the newly constructed database GAUDI-DI. Experimental results show that the proposed method is superior to the state-of-the-art method.
【8】 Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation
标题：MuseForm：音乐生成的细粒度和粗粒度关注的转换器
链接：https://arxiv.org/abs/2210.10349
* 与cs.SD语音【4】为同一篇
作者：Botao Yu,Peiling Lu,Rui Wang,Wei Hu,Xu Tan,Wei Ye,Shikun Zhang,Tao Qin,Tie-Yan Liu机构：†State Key Laboratory for Novel Software Technology, Nanjing University, China, ‡Microsoft Research Asia, §National Engineering Research Center for Software Engineering, Peking University, China备注：Accepted by the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022)摘要：符号音乐生成的目的是自动生成乐谱。最近的趋势是在音乐生成中使用Transformer或其变体，然而，这是次优的，因为全部注意力不能有效地对通常较长的音乐序列（例如，超过10，000个记号），并且现有模型在生成音乐重复结构方面存在缺点。本文提出了一种新的基于细粒度和粗粒度注意力的音乐生成Transformer Museformer。具体地，利用细粒度关注，特定小节的表征直接关注与音乐结构最相关的小节的所有表征（例如，通过相似性统计选择的先前的第一、第二、第四和第八条）;通过粗粒度的关注，令牌只关注其他条的汇总，而不是它们的每个令牌，从而降低了计算成本。其优点有两方面。首先，它可以通过细粒度的注意力捕捉与音乐结构相关的相关性，并通过粗粒度的注意力捕捉其他上下文信息。第二，它是高效的，与全注意力对应物相比，它可以建模超过3倍长的音乐序列。客观和主观实验结果表明，该算法能够生成高质量、结构较好的长音乐序列。摘要：Symbolic music generation aims to generate music scores automatically. A recent trend is to use Transformer or its variants in music generation, which is, however, suboptimal, because the full attention cannot efficiently model the typically long music sequences (e.g., over 10,000 tokens), and the existing models have shortcomings in generating musical repetition structures. In this paper, we propose Museformer, a Transformer with a novel fine- and coarse-grained attention for music generation. Specifically, with the fine-grained attention, a token of a specific bar directly attends to all the tokens of the bars that are most relevant to music structures (e.g., the previous 1st, 2nd, 4th and 8th bars, selected via similarity statistics); with the coarse-grained attention, a token only attends to the summarization of the other bars rather than each token of them so as to reduce the computational cost. The advantages are two-fold. First, it can capture both music structure-related correlations via the fine-grained attention, and other contextual information via the coarse-grained attention. Second, it is efficient and can model over 3X longer music sequences compared to its full-attention counterpart. Both objective and subjective experimental results demonstrate its ability to generate long music sequences with high quality and better structures.
【9】 Two-stage training method for Japanese electrolaryngeal speech enhancement based on sequence-to-sequence voice conversion
标题：一种基于序列到序列语音转换的日语喉电语音增强两阶段训练方法
链接：https://arxiv.org/abs/2210.10314
* 与cs.SD语音【5】为同一篇
作者：Ding Ma,Lester Phillip Violeta,Kazuhiro Kobayashi,Tomoki Toda机构：Nagoya University, Japan摘要：序列到序列（seq2seq）语音转换（VC）模型与传统VC模型相比在将电喉（EL）语音转换为正常语音（EL2SP）方面具有更大的潜力。然而，基于seq2seq VC的EL2SP算法需要足够大的并行数据量进行模型训练，当训练数据量不足时，EL2SP算法的性能会显著下降。针对这一问题，提出了一种新的两阶段策略，在并行数据量较少的情况下，对基于seq2seq VC的EL2SP进行性能优化.与以往研究中使用高质量数据增强不同，我们首先将大量不完美的EL和正常语音的合成并行数据与原始数据集结合到VC训练中。然后，仅使用原始并行数据集进行第二阶段训练。实验结果表明，该方法逐步提高了基于seq2seq VC的EL2SP算法的性能。摘要：Sequence-to-sequence (seq2seq) voice conversion (VC) models have greater potential in converting electrolaryngeal (EL) speech to normal speech (EL2SP) compared to conventional VC models. However, EL2SP based on seq2seq VC requires a sufficiently large amount of parallel data for the model training and it suffers from significant performance degradation when the amount of training data is insufficient. To address this issue, we suggest a novel, two-stage strategy to optimize the performance on EL2SP based on seq2seq VC when a small amount of the parallel dataset is available. In contrast to utilizing high-quality data augmentations in previous studies, we first combine a large amount of imperfect synthetic parallel data of EL and normal speech, with the original dataset into VC training. Then, a second stage training is conducted with the original parallel dataset only. The results show that the proposed method progressively improves the performance of EL2SP based on seq2seq VC.
【10】 A Data-Driven Investigation of Noise-Adaptive Utterance Generation with Linguistic Modification
标题：基于数据驱动的语言修改噪声自适应语音生成研究
链接：https://arxiv.org/abs/2210.10252
* 与cs.SD语音【6】为同一篇
作者：Anupama Chingacham,Vera Demberg,Dietrich Klakow机构：Saarland Informatics Campus, Saarland University, Germany, ©Copyright , IEEE. Published in the , IEEE Spoken Language Technology Workshop (SLT) (SLT ,)摘要：在嘈杂的环境中，人类可能难以理解语音。口头对话系统可以帮助增强其输出的可理解性，这或者通过修改语音合成（例如，模仿伦巴第语）或通过优化语言生成。我们在这里关注的是第二种方法，通过这种方法，一个预期的信息是用在特定的嘈杂环境中更容易理解的单词来实现的。通过进行语音感知实验，我们创建了一个由900个释义的数据集，这些释义由听力正常的英语母语者感知。我们发现，仔细选择释义可以在SNR为-5dB时提高可懂度33%。我们对数据的分析表明，释义之间的可懂度差异主要是由噪声鲁棒的声学线索驱动的。在此基础上，提出了一种可懂度感知的释义排序模型，在信噪比为-5dB时，该模型的性能相对于基准模型提高了31.37%。摘要：In noisy environments, speech can be hard to understand for humans. Spoken dialog systems can help to enhance the intelligibility of their output, either by modifying the speech synthesis (e.g., imitate Lombard speech) or by optimizing the language generation. We here focus on the second type of approach, by which an intended message is realized with words that are more intelligible in a specific noisy environment. By conducting a speech perception experiment, we created a dataset of 900 paraphrases in babble noise, perceived by native English speakers with normal hearing. We find that careful selection of paraphrases can improve intelligibility by 33% at SNR -5 dB. Our analysis of the data shows that the intelligibility differences between paraphrases are mainly driven by noise-robust acoustic cues. Furthermore, we propose an intelligibility-aware paraphrase ranking model, which outperforms baseline models with a relative improvement of 31.37% at SNR -5 dB.
【11】 Speaker- and Age-Invariant Training for Child Acoustic Modeling Using Adversarial Multi-Task Learning
标题：使用对抗性多任务学习的儿童声学建模的说话人和年龄不变的训练
链接：https://arxiv.org/abs/2210.10231
* 与cs.SD语音【7】为同一篇
作者：Mostafa Shahin,Beena Ahmed,Julien Epps机构：School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, Australia备注：Submitted to ICASSP2023摘要：儿童语音声学建模的主要挑战之一是儿童的发音器官随着他们的成长而发生的快速变化、他们不同的生长速率以及随后在同一年龄组中的高可变性。这些高的声学变化以及儿童语音语料库的缺乏阻碍了可靠的儿童语音识别系统的开发。本文提出了一种基于对抗式多任务学习的说话人和年龄不变的训练方法。该系统由一个生成器共享网络组成，该网络学习生成说话人和年龄不变特征，并连接到三个区分网络，分别用于音素、年龄和说话人。该发生器网络被训练成以对抗性多任务学习方式最小化音素辨别损失并最大化说话人和年龄辨别损失。发生器网络是一个时间延迟神经网络（TDNN）结构，而三个鉴别器是前馈网络。将该系统应用于OGI语音库，ASR的WER降低了13%。摘要：One of the major challenges in acoustic modelling of child speech is the rapid changes that occur in the children's articulators as they grow up, their differing growth rates and the subsequent high variability in the same age group. These high acoustic variations along with the scarcity of child speech corpora have impeded the development of a reliable speech recognition system for children. In this paper, a speaker- and age-invariant training approach based on adversarial multi-task learning is proposed. The system consists of one generator shared network that learns to generate speaker- and age-invariant features connected to three discrimination networks, for phoneme, age, and speaker. The generator network is trained to minimize the phoneme-discrimination loss and maximize the speaker- and age-discrimination losses in an adversarial multi-task learning fashion. The generator network is a Time Delay Neural Network (TDNN) architecture while the three discriminators are feed-forward networks. The system was applied to the OGI speech corpora and achieved a 13% reduction in the WER of the ASR.
【12】 BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds
标题：鸟声去噪：鸟鸣的深度视觉去噪
链接：https://arxiv.org/abs/2210.10196
* 与cs.SD语音【8】为同一篇
作者：Youshan Zhang,Jialu Li机构：Yeshiva University, NYC, NY, Cornell University, Ithaca, NY摘要：音频去噪已经使用传统和基于深度学习的方法进行了几十年的探索。然而，这些方法仍然局限于手动添加的人工噪声或较低的去噪音频质量。为了克服这些挑战，我们收集了大规模自然噪声鸟声数据集。我们首次将音频去噪问题转化为图像分割问题，提出了一种深度视觉音频去噪（DVAD）模型。我们开发了一个音频ImageMask工具，并建议使用一个Few-Shot泛化策略来标记这些图像。大量实验结果表明，该模型具有较好的性能。我们还表明，我们的方法可以很容易地推广到语音去噪，音频分离，音频增强，和噪声估计。摘要：Audio denoising has been explored for decades using both traditional and deep learning-based methods. However, these methods are still limited to either manually added artificial noise or lower denoised audio quality. To overcome these challenges, we collect a large-scale natural noise bird sound dataset. We are the first to transfer the audio denoising problem into an image segmentation problem and propose a deep visual audio denoising (DVAD) model. With a total of 14,120 audio images, we develop an audio ImageMask tool and propose to use a few-shot generalization strategy to label these images. Extensive experimental results demonstrate that the proposed model achieves state-of-the-art performance. We also show that our method can be easily generalized to speech denoising, audio separation, audio enhancement, and noise estimation.
【13】 Simple and Effective Unsupervised Speech Translation
标题：简单有效的无监督语音翻译
链接：https://arxiv.org/abs/2210.10191
* 与cs.SD语音【9】为同一篇
作者：Changhan Wang,Hirofumi Inaguma,Peng-Jen Chen,Ilia Kulikov,Yun Tang,Wei-Ning Hsu,Michael Auli,Juan Pino机构：Meta - Fundamental AI Research (FAIR)摘要：对于大多数语言来说，用于训练语音任务模型的标记数据量是有限的，然而，对于需要覆盖两种不同语言的标记数据的语音翻译来说，数据稀缺性加剧。为了解决这个问题，我们研究了一种简单有效的方法，利用无监督语音识别、机器翻译和语音合成的最新进展，在没有标记数据的情况下构建语音翻译系统，或者在流水线方法中，或者生成用于训练端到端语音翻译模型的伪标记。此外，我们提出了一种用于预训练语音模型的无监督域自适应技术，该技术提高了下游无监督语音识别的性能，特别是在低资源设置下。实验表明，在Libri-Trans基准测试中，无监督语音到文本翻译的性能比以前的无监督技术水平高3.2 BLEU，在CoVoST 2上，我们的最佳系统在五个X-En方向上的平均性能比仅两年前的最佳有监督端到端模型（无预训练）高5.0 BLEU。我们还报告了在Must-C和CVSS基准测试中的竞争结果。摘要：The amount of labeled data to train models for speech tasks is limited for most languages, however, the data scarcity is exacerbated for speech translation which requires labeled data covering two different languages. To address this issue, we study a simple and effective approach to build speech translation systems without labeled data by leveraging recent advances in unsupervised speech recognition, machine translation and speech synthesis, either in a pipeline approach, or to generate pseudo-labels for training end-to-end speech translation models. Furthermore, we present an unsupervised domain adaptation technique for pre-trained speech models which improves the performance of downstream unsupervised speech recognition, especially for low-resource settings. Experiments show that unsupervised speech-to-text translation outperforms the previous unsupervised state of the art by 3.2 BLEU on the Libri-Trans benchmark, on CoVoST 2, our best systems outperform the best supervised end-to-end models (without pre-training) from only two years ago by an average of 5.0 BLEU over five X-En directions. We also report competitive results on MuST-C and CVSS benchmarks.
机器翻译，仅供参考
每日语音速递