今日论文合集：cs.SD语音3篇，eess.AS音频处理5篇。【1】 An investigation of the reconstruction capacity of stacked convolutional autoencoders for log-mel-spectrograms
标题：LOG-Mel谱图叠加式卷积编码器重建能力的研究
链接：http://arxiv.org/pdf/2301.07665v1
作者：Anastasia Natsiou,Luca Longo,Sean O'Leary机构：Technological University of Dublin, Dublin, Ireland, Se´an O’Leary摘要：在音频处理应用中，基于高级表示的富有表现力的声音的生成表现出很高的需求。这些表征可以用来操纵音色和影响创造性乐器音符的合成。诸如神经网络的现代算法已经激发了基于乐器音色压缩的表现力合成器的发展。无监督深度学习方法可以通过训练网络学习从波形或频谱图到低维表示的映射来实现音频压缩。本研究调查使用堆叠式卷积自动编码器来压缩各种乐器的单一音高的时频音频表示。进一步探索超参数和正则化技术，以提高性能的初始设计。在无监督的方式下，该网络能够基于潜在表示来重构单声道和谐波声音。此外，我们还引入了一个评价指标来衡量原始样本和重构样本之间的相似性。评估声音合成的深层生成模型是一项具有挑战性的任务。我们的方法基于所产生频率的准确性，因为它是感知和声的重要指标。这项工作有望加速未来使用神经自动编码器进行音频压缩的实验。摘要：In audio processing applications, the generation of expressive sounds based on high-level representations demonstrates a high demand. These representations can be used to manipulate the timbre and influence the synthesis of creative instrumental notes. Modern algorithms, such as neural networks, have inspired the development of expressive synthesizers based on musical instrument timbre compression. Unsupervised deep learning methods can achieve audio compression by training the network to learn a mapping from waveforms or spectrograms to low-dimensional representations. This study investigates the use of stacked convolutional autoencoders for the compression of time-frequency audio representations for a variety of instruments for a single pitch. Further exploration of hyper-parameters and regularization techniques is demonstrated to enhance the performance of the initial design. In an unsupervised manner, the network is able to reconstruct a monophonic and harmonic sound based on latent representations. In addition, we introduce an evaluation metric to measure the similarity between the original and reconstructed samples. Evaluating a deep generative model for the synthesis of sound is a challenging task. Our approach is based on the accuracy of the generated frequencies as it presents a significant metric for the perception of harmonic sounds. This work is expected to accelerate future experiments on audio compression using neural autoencoders.
【2】 The Newsbridge -Telecom SudParis VoxCeleb Speaker Recognition Challenge 2022 System Description
标题：Newsbridge-Telecom suParis VoxCeleb说话人识别挑战赛2022系统描述
链接：http://arxiv.org/pdf/2301.07491v1
作者：Yannis Tevissen,Jérôme Boudy,Frédéric Petitpont机构：SAMOVAR, Telecom SudParis, Institut Polytechnique de Paris, France, Newsbridge, France摘要：我们在说话人日记化赛道中描述了我们团队在2022年VoxCeleb说话人识别挑战赛（VoxSRC 2022）中使用的系统。我们的解决方案是围绕一种新的语音活动检测算法组合而设计的，该算法利用了多个系统的优势。提出了一种基于分类器熵的多流分类方法。我们称这种方法为多流语音活动检测，并将其与标准基线二值化嵌入、聚类和再分割一起使用。通过这项工作，我们成功地证明，使用一个强大的基线和只工作在语音活动检测，可以实现接近最先进的结果。摘要：We describe the system used by our team for the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC 2022) in the speaker diarization track. Our solution was designed around a new combination of voice activity detection algorithms that uses the strengths of several systems. We introduce a novel multi stream approach with a decision protocol based on classifiers entropy. We called this method a multi-stream voice activity detection and used it with standard baseline diarization embeddings, clustering and resegmentation. With this work, we successfully demonstrated that using a strong baseline and working only on voice activity detection, one can achieved close to state-of-theart results.
【3】 Towards Voice Reconstruction from EEG during Imagined Speech
标题：基于想象语音的脑电语音重建方法研究
链接：http://arxiv.org/pdf/2301.07173v1
作者：Young-Eun Lee,Seo-Hyun Lee,Sang-Ho Kim,Seong-Whan Lee机构：Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea, Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea备注：9 pages, 4 figures, accepted paper of AAAI 2023 in main track摘要：将大脑活动中的想象言语转换为语音是一个具有挑战性和吸引力的研究课题，它可以通过大脑信号提供新的人类交流手段。通过使用口头语音数据的侵入性测量，从大脑活动重构语音的努力已经显示出其潜力，然而，在重构想象语音方面面临挑战。在本文中，我们提出了NeuroTalk，它将想象的语音的非侵入性脑信号转换为用户自己的声音。我们的模型是用口语EEG训练的，口语EEG被推广以适应想象语音的域，从而允许想象语音和作为地面事实的声音之间的自然对应。在我们的架构中，自动语音识别解码器有助于分解所产生语音的音素，从而显示出从看不见的词重建语音的潜力。我们的结果暗示了从人类脑电信号合成语音的潜力，不仅从口头语音，而且从想象语音的脑信号。摘要：Translating imagined speech from human brain activity into voice is a challenging and absorbing research issue that can provide new means of human communication via brain signals. Endeavors toward reconstructing speech from brain activity have shown their potential using invasive measures of spoken speech data, however, have faced challenges in reconstructing imagined speech. In this paper, we propose NeuroTalk, which converts non-invasive brain signals of imagined speech into the user's own voice. Our model was trained with spoken speech EEG which was generalized to adapt to the domain of imagined speech, thus allowing natural correspondence between the imagined speech and the voice as a ground truth. In our framework, automatic speech recognition decoder contributed to decomposing the phonemes of generated speech, thereby displaying the potential of voice reconstruction from unseen words. Our results imply the potential of speech synthesis from human EEG signals, not only from spoken speech but also from the brain signals of imagined speech.
【1】 Dereverberation in Acoustic Sensor Networks Using Weighted Prediction Error With Microphone-dependent Prediction Delays
标题：基于带麦克风相关预测延迟的加权预测误差的声传感器网络混响消除
链接：http://arxiv.org/pdf/2301.07649v1
作者：Anselm Lohmann,Toon van Waterschoot,Joerg Bitzer,Simon Doclo机构：University of Oldenburg, Dept. of Medical Physics and Acoustics, Oldenburg, Germany, KU Leuven, Department of Electrical Engineering (ESAT-STADIUS), Leuven, Belgium, Fraunhofer IDMT, Project Group Hearing, Speech and Audio Technology, Oldenburg, Germany摘要：在过去的几十年里，人们提出了多种多麦克风语音去混响算法，其中加权预测误差（WPE）算法就是其中之一。在WPE算法中，需要预测延迟来降低预测信号与参考麦克风信号中的直接分量之间的相关性。在具有紧密间隔的麦克风的紧凑阵列中，预测延迟通常被选择为与麦克风无关。在具有空间分布的麦克风的声学传感器网络中，参考麦克风和其他麦克风之间的语音源的大到达时间差（TDOA）通常发生。因此，当使用与麦克风无关的预测延迟时，参考信号和预测信号可能仍然显著相关，从而导致去混响输出信号中的失真。为了解相关信号，在本文中，我们提出应用相对于参考麦克风的TDOA补偿，导致WPE算法的麦克风相关预测延迟。我们考虑在短时傅立叶变换域中使用交叉带滤波的最优TDOA补偿以及带间和整数延迟近似。使用oracle以及估计的TDOA针对不同混响时间的仿真结果清楚地示出了使用麦克风相关预测延迟的益处。摘要：In the last decades several multi-microphone speech dereverberation algorithms have been proposed, among which the weighted prediction error (WPE) algorithm. In the WPE algorithm, a prediction delay is required to reduce the correlation between the prediction signals and the direct component in the reference microphone signal. In compact arrays with closely-spaced microphones, the prediction delay is often chosen microphone-independent. In acoustic sensor networks with spatially distributed microphones, large time-differences-of-arrival (TDOAs) of the speech source between the reference microphone and other microphones typically occur. Hence, when using a microphone-independent prediction delay the reference and prediction signals may still be significantly correlated, leading to distortion in the dereverberated output signal. In order to decorrelate the signals, in this paper we propose to apply TDOA compensation with respect to the reference microphone, resulting in microphone-dependent prediction delays for the WPE algorithm. We consider both optimal TDOA compensation using crossband filtering in the short-time Fourier transform domain as well as band-to-band and integer delay approximations. Simulation results for different reverberation times using oracle as well as estimated TDOAs clearly show the benefit of using microphone-dependent prediction delays.
【2】 Towards Voice Reconstruction from EEG during Imagined Speech
标题：基于想象语音的脑电语音重建方法研究
链接：http://arxiv.org/pdf/2301.07173v1
*与cs.SD语音【3】为同一篇
作者：Young-Eun Lee,Seo-Hyun Lee,Sang-Ho Kim,Seong-Whan Lee机构：Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea, Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea备注：9 pages, 4 figures, accepted paper of AAAI 2023 in main track摘要：将大脑活动中的想象言语转换为语音是一个具有挑战性和吸引力的研究课题，它可以通过大脑信号提供新的人类交流手段。通过使用口头语音数据的侵入性测量，从大脑活动重构语音的努力已经显示出其潜力，然而，在重构想象语音方面面临挑战。在本文中，我们提出了NeuroTalk，它将想象的语音的非侵入性脑信号转换为用户自己的声音。我们的模型是用口语EEG训练的，口语EEG被推广以适应想象语音的域，从而允许想象语音和作为地面事实的声音之间的自然对应。在我们的架构中，自动语音识别解码器有助于分解所产生语音的音素，从而显示出从看不见的词重建语音的潜力。我们的结果暗示了从人类脑电信号合成语音的潜力，不仅从口头语音，而且从想象语音的脑信号。摘要：Translating imagined speech from human brain activity into voice is a challenging and absorbing research issue that can provide new means of human communication via brain signals. Endeavors toward reconstructing speech from brain activity have shown their potential using invasive measures of spoken speech data, however, have faced challenges in reconstructing imagined speech. In this paper, we propose NeuroTalk, which converts non-invasive brain signals of imagined speech into the user's own voice. Our model was trained with spoken speech EEG which was generalized to adapt to the domain of imagined speech, thus allowing natural correspondence between the imagined speech and the voice as a ground truth. In our framework, automatic speech recognition decoder contributed to decomposing the phonemes of generated speech, thereby displaying the potential of voice reconstruction from unseen words. Our results imply the potential of speech synthesis from human EEG signals, not only from spoken speech but also from the brain signals of imagined speech.
【3】 An investigation of the reconstruction capacity of stacked convolutional autoencoders for log-mel-spectrograms
标题：LOG-Mel谱图叠加式卷积编码器重建能力的研究
链接：http://arxiv.org/pdf/2301.07665v1
*与cs.SD语音【1】为同一篇
作者：Anastasia Natsiou,Luca Longo,Sean O'Leary机构：Technological University of Dublin, Dublin, Ireland, Se´an O’Leary摘要：在音频处理应用中，基于高级表示的富有表现力的声音的生成表现出很高的需求。这些表征可以用来操纵音色和影响创造性乐器音符的合成。诸如神经网络的现代算法已经激发了基于乐器音色压缩的表现力合成器的发展。无监督深度学习方法可以通过训练网络学习从波形或频谱图到低维表示的映射来实现音频压缩。本研究调查使用堆叠式卷积自动编码器来压缩各种乐器的单一音高的时频音频表示。进一步探索超参数和正则化技术，以提高性能的初始设计。在无监督的方式下，该网络能够基于潜在表示来重构单声道和谐波声音。此外，我们还引入了一个评价指标来衡量原始样本和重构样本之间的相似性。评估声音合成的深层生成模型是一项具有挑战性的任务。我们的方法基于所产生频率的准确性，因为它是感知和声的重要指标。这项工作有望加速未来使用神经自动编码器进行音频压缩的实验。摘要：In audio processing applications, the generation of expressive sounds based on high-level representations demonstrates a high demand. These representations can be used to manipulate the timbre and influence the synthesis of creative instrumental notes. Modern algorithms, such as neural networks, have inspired the development of expressive synthesizers based on musical instrument timbre compression. Unsupervised deep learning methods can achieve audio compression by training the network to learn a mapping from waveforms or spectrograms to low-dimensional representations. This study investigates the use of stacked convolutional autoencoders for the compression of time-frequency audio representations for a variety of instruments for a single pitch. Further exploration of hyper-parameters and regularization techniques is demonstrated to enhance the performance of the initial design. In an unsupervised manner, the network is able to reconstruct a monophonic and harmonic sound based on latent representations. In addition, we introduce an evaluation metric to measure the similarity between the original and reconstructed samples. Evaluating a deep generative model for the synthesis of sound is a challenging task. Our approach is based on the accuracy of the generated frequencies as it presents a significant metric for the perception of harmonic sounds. This work is expected to accelerate future experiments on audio compression using neural autoencoders.
【4】 The Newsbridge -Telecom SudParis VoxCeleb Speaker Recognition Challenge 2022 System Description
标题：Newsbridge-Telecom suParis VoxCeleb说话人识别挑战赛2022系统描述
链接：http://arxiv.org/pdf/2301.07491v1
*与cs.SD语音【2】为同一篇
作者：Yannis Tevissen,Jérôme Boudy,Frédéric Petitpont机构：SAMOVAR, Telecom SudParis, Institut Polytechnique de Paris, France, Newsbridge, France摘要：我们在说话人日记化赛道中描述了我们团队在2022年VoxCeleb说话人识别挑战赛（VoxSRC 2022）中使用的系统。我们的解决方案是围绕一种新的语音活动检测算法组合而设计的，该算法利用了多个系统的优势。提出了一种基于分类器熵的多流分类方法。我们称这种方法为多流语音活动检测，并将其与标准基线二值化嵌入、聚类和再分割一起使用。通过这项工作，我们成功地证明，使用一个强大的基线和只工作在语音活动检测，可以实现接近最先进的结果。摘要：We describe the system used by our team for the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC 2022) in the speaker diarization track. Our solution was designed around a new combination of voice activity detection algorithms that uses the strengths of several systems. We introduce a novel multi stream approach with a decision protocol based on classifiers entropy. We called this method a multi-stream voice activity detection and used it with standard baseline diarization embeddings, clustering and resegmentation. With this work, we successfully demonstrated that using a strong baseline and working only on voice activity detection, one can achieved close to state-of-theart results.
【5】 Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretraining
标题：通过多语言微调和持续的预训练使多语言语音表示模型适应新的、资源不足的语言
链接：http://arxiv.org/pdf/2301.07295v1
作者：Karol Nowakowski,Michal Ptaszynski,Kyoko Murasaki,Jagna Nieuważny机构：Tohoku University of Community Service and Science, Sakata, Yamagata, Japan, Kitami Institute of Technology, Kitami, Hokkaido, Japan, Yokohama National University (Professor Emeritus), Yokohama, Kanagawa, Japan, A R T I C L E I N F OJournal-ref：Information Processing & Management, Volume 60, Issue 2, March 2023, 103148, ISSN 0306-4573摘要：近年来，在大规模多语言文本或语音数据上通过自监督预训练学习的神经模型对于资源不足的语言表现出了良好的效果，特别是当相关语言的数据量相对较大时。虽然该技术有可能促进语言文档项目中执行的任务，如语音转录，但从头开始为每种新语言预先训练多语言模型是非常不切实际的。我们研究了将现有的多语言wav 2 vec 2.0模型应用于新语言的可能性，重点是来自一种极度濒危语言的实际实地调查数据：阿伊努具体地说，我们（i）检查在微调中利用来自类似语言的数据的可行性;（ii）验证是否可以通过对目标语言数据的进一步预训练来改进模型的性能。我们的结果表明，持续的预训练是使wav 2 vec 2.0模型适应新语言的最有效的方法，并导致错误率的显著降低。此外，我们发现，如果一个预先训练的模型是针对一个相关的语音变体或一个具有相似语音特征的不相关的语言，那么当目标语言中的标记数据很少时，使用来自该语言的额外数据进行多语言微调可以对语音识别性能产生积极的影响。摘要：In recent years, neural models learned through self-supervised pretraining on large scale multilingual text or speech data have exhibited promising results for underresourced languages, especially when a relatively large amount of data from related language(s) is available. While the technology has a potential for facilitating tasks carried out in language documentation projects, such as speech transcription, pretraining a multilingual model from scratch for every new language would be highly impractical. We investigate the possibility for adapting an existing multilingual wav2vec 2.0 model for a new language, focusing on actual fieldwork data from a critically endangered tongue: Ainu. Specifically, we (i) examine the feasibility of leveraging data from similar languages also in fine-tuning; (ii) verify whether the model's performance can be improved by further pretraining on target language data. Our results show that continued pretraining is the most effective method to adapt a wav2vec 2.0 model for a new language and leads to considerable reduction in error rates. Furthermore, we find that if a model pretrained on a related speech variety or an unrelated language with similar phonological characteristics is available, multilingual fine-tuning using additional data from that language can have positive impact on speech recognition performance when there is very little labeled data in the target language.
机器翻译，仅供参考
每日语音速递