今日论文合集：cs.SD语音7篇，eess.AS音频处理11篇。cs.SD语音【1】 Controllable Music Production with Diffusion Models and Guidance Gradients标题：具有扩散模型和引导梯度的可控音乐制作链接：https://arxiv.org/abs/2311.00613作者：Mark Levy,Bruno Di Giorgi,Floris Weers,Angelos Katharopoulos,Tom Nickson摘要：我们演示了如何从扩散模型的条件生成可以用来解决各种现实的任务，在44.1kHz的立体声音频与采样时间的指导下的音乐生产。我们考虑的场景包括音乐音频的延续，修复和再生，在两个不同的音乐曲目之间创建平滑过渡，以及将所需的风格特征转移到现有的音频片段。我们通过在采样时在一个简单的框架中应用指导来实现这一点，该框架支持重建和分类损失，或两者的任何组合。这种方法确保生成的音频可以匹配其周围的上下文，或者符合相对于任何合适的预先训练的分类器或嵌入模型指定的类分布或潜在表示。摘要：We demonstrate how conditional generation from diffusion models can be used to tackle a variety of realistic tasks in the production of music in 44.1kHz stereo audio with sampling-time guidance. The scenarios we consider include continuation, inpainting and regeneration of musical audio, the creation of smooth transitions between two different music tracks, and the transfer of desired stylistic characteristics to existing audio clips. We achieve this by applying guidance at sampling time in a simple framework that supports both reconstruction and classification losses, or any combination of the two. This approach ensures that generated audio can match its surrounding context, or conform to a class distribution or latent representation specified relative to any suitable pre-trained classifier or embedding model.【2】 Active Noise Control Portable Device Design标题：有源噪声控制便携式装置的设计链接：https://arxiv.org/abs/2311.00535作者：kai Wu,Yuanyuan Chen摘要：虽然我们的世界充满了我们无法抗拒享受的自然声音，但它也充满了其他可能令人恼火的声音，这就是噪音。噪声不仅影响工作效率，而且影响人体健康。降低噪声是一个非常重要和困难的问题。多年来，这个问题一直在以多种方式解决。现有的降噪方法大多依赖于材料和传输介质，仅对高频噪声有一定的抑制作用。然而，有效的降噪方法，特别是对低频噪声的降噪方法非常有限。 在这里，我们提出了一个降噪系统，包括一个传感器来检测环境中的噪声。然后，噪声将被发送到电子控制系统以处理噪声，电子控制系统将产生反相频率信号以抵消干扰。最后，处理后的较小噪声将由扬声器广播。通过这种智能降噪系统，即使是低频噪音也可以消除。 该系统还集成了睡眠跟踪和音乐播放器应用程序。它还可以记忆和存储相同环境的设置，感知温度，以及智能控制家居家具，火灾报警等，这个智能系统可以通过Wi-Fi或蓝牙轻松传输数据并通过其APP进行控制。 在本项目中，我们将展示上述技术的一个模型，该模型可用于各种环境，以防止噪音污染，并为难以找到安静的睡眠，工作或学习环境的人提供解决方案。摘要：While our world is filled with its own natural sounds that we can't resist enjoying, it is also chock-full of other sounds that can be irritating, this is noise. Noise not only influences the working efficiency but also the human's health. The problem of reducing noise is one of great importance and great difficulty. The problem has been addressed in many ways over the years. The current methods for noise reducing mostly rely on the materials and transmission medium, which are only effective to some extent for the high frequency noise. However, the effective reduction noise method especially for low frequency noise is very limited. Here we come up with a noise reduction system consist of a sensor to detect the noise in the environment. Then the noise will be sent to an electronic control system to process the noise, which will generate a reverse phase frequency signal to counteract the disturbance. Finally, the processed smaller noise will be broadcasted by the speaker. Through this smart noise reduction system, even the noise with low-frequency can be eliminated. The system is also integrated with sleep tracking and music player applications. It can also remember and store settings for the same environment, sense temperature, and smart control of home furniture, fire alarm, etc. This smart system can transfer data easily by Wi-Fi or Bluetooth and controlled by its APP. In this project, we will present a model of the above technology which can be used in various environments to prevent noise pollution and provide a solution to the people who have difficulties finding a peaceful and quiet environment for sleep, work or study.【3】 Deep Neural Networks for Automatic Speaker Recognition Do Not Learn Supra-Segmental Temporal Features标题：用于说话人自动识别的深度神经网络不学习超音段时间特征链接：https://arxiv.org/abs/2311.00489作者：Daniel Neururer,Volker Dellow,Thilo Stadelmann备注：7 pages, 2 figures, 3 tables摘要：虽然深度神经网络在自动说话人识别和相关任务中显示出令人印象深刻的结果，但令人不满的是，人们对这些结果的确切原因知之甚少。在先前的工作中，部分成功归因于他们对超节段时间信息（SST）建模的能力，即，除了频谱特征之外，还学习语音节奏韵律特征。在本文中，我们（i）提出并应用一种新的测试来量化最先进的说话人识别神经网络的性能在多大程度上可以通过对SST建模来解释;（ii）提出几种方法来迫使各自的网络更多地关注SST并评估其优点。我们发现，各种基于CNN和RNN的神经网络架构的说话人识别没有模型SST到任何足够的程度，即使是被迫的。这些结果为未来更好地利用完整语音信号的有影响力的研究提供了高度相关的基础，并深入了解了此类网络的内部工作原理，增强了语音技术深度学习的可解释性。摘要：While deep neural networks have shown impressive results in automatic speaker recognition and related tasks, it is dissatisfactory how little is understood about what exactly is responsible for these results. Part of the success has been attributed in prior work to their capability to model supra-segmental temporal information (SST), i.e., learn rhythmic-prosodic characteristics of speech in addition to spectral features. In this paper, we (i) present and apply a novel test to quantify to what extent the performance of state-of-the-art neural networks for speaker recognition can be explained by modeling SST; and (ii) present several means to force respective nets to focus more on SST and evaluate their merits. We find that a variety of CNN- and RNN-based neural network architectures for speaker recognition do not model SST to any sufficient degree, even when forced. The results provide a highly relevant basis for impactful future research into better exploitation of the full speech signal and give insights into the inner workings of such networks, enhancing explainability of deep learning for speech technologies.【4】 Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling标题：Distil-Whisper：基于大规模伪标注的稳健知识提取链接：https://arxiv.org/abs/2311.00430作者：Sanchit Gandhi,Patrick von Platen,Alexander M. Rush备注：30 pages, 2 figures, 25 tables摘要：随着预先训练的语音识别模型的大小增加，在低延迟或资源受限的环境中运行这些大型模型变得具有挑战性。在这项工作中，我们利用伪标签来组装一个大规模的开源数据集，我们使用它来将Whisper模型提取成一个较小的变体，称为Distil-Whisper。使用简单的单词错误率（WER）启发式算法，我们只选择最高质量的伪标签进行训练。蒸馏模型的速度提高了5.8倍，参数减少了51%，同时在zero-shot传输设置中对分布外测试数据的WER在1%以内。Distil-Whisper保持了Whisper模型对困难声学条件的鲁棒性，同时在长格式音频上不太容易出现幻觉错误。Distil-Whisper旨在与Whisper配对进行推测性解码，产生2倍的加速，同时在数学上确保与原始模型相同的输出。为了促进这一领域的进一步研究，我们公开了我们的训练代码、推理代码和模型。摘要：As the size of pre-trained speech recognition models increases, running these large models in low-latency or resource-constrained environments becomes challenging. In this work, we leverage pseudo-labelling to assemble a large-scale open-source dataset which we use to distill the Whisper model into a smaller variant, called Distil-Whisper. Using a simple word error rate (WER) heuristic, we select only the highest quality pseudo-labels for training. The distilled model is 5.8 times faster with 51% fewer parameters, while performing to within 1% WER on out-of-distribution test data in a zero-shot transfer setting. Distil-Whisper maintains the robustness of the Whisper model to difficult acoustic conditions, while being less prone to hallucination errors on long-form audio. Distil-Whisper is designed to be paired with Whisper for speculative decoding, yielding a 2 times speed-up while mathematically ensuring the same outputs as the original model. To facilitate further research in this domain, we make our training code, inference code and models publicly accessible.【5】 Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables标题：语义听觉：用双耳可听物编程声学场景链接：https://arxiv.org/abs/2311.00320作者：Bandhav Veluri,Malek Itani,Justin Chan,Takuya Yoshioka,Shyamnath Gollakota摘要：想象一下，能够在公园里听鸟儿啁啾，而不会听到其他徒步旅行者的喋喋不休，或者能够在繁忙的街道上屏蔽交通噪音，同时仍然能够听到紧急警报器和汽车喇叭声。我们引入语义听觉，这是一种新的听觉设备功能，使它们能够实时关注或忽略来自现实环境的特定声音，同时还保留了空间线索。为了实现这一目标，我们做出了两项技术贡献：1）我们提出了第一个神经网络，可以在存在干扰声音和背景噪声的情况下实现双耳目标声音提取，2）我们设计了一种训练方法，使我们的系统能够推广到现实世界中使用。结果表明，我们的系统可以运行20个声音类，我们基于变压器的网络在连接的智能手机上的运行时间为6.56 ms。在以前看不见的室内和室外场景中对参与者进行的野外评估表明，我们的概念验证系统可以提取目标声音并概括以保留其双耳输出中的空间线索。项目页面代码：https://semantichearing.cs.washington.edu摘要：Imagine being able to listen to the birds chirping in a park without hearing the chatter from other hikers, or being able to block out traffic noise on a busy street while still being able to hear emergency sirens and car honks. We introduce semantic hearing, a novel capability for hearable devices that enables them to, in real-time, focus on, or ignore, specific sounds from real-world environments, while also preserving the spatial cues. To achieve this, we make two technical contributions: 1) we present the first neural network that can achieve binaural target sound extraction in the presence of interfering sounds and background noise, and 2) we design a training methodology that allows our system to generalize to real-world use. Results show that our system can operate with 20 sound classes and that our transformer-based network has a runtime of 6.56 ms on a connected smartphone. In-the-wild evaluation with participants in previously unseen indoor and outdoor scenarios shows that our proof-of-concept system can extract the target sounds and generalize to preserve the spatial cues in its binaural output. Project page with code: https://semantichearing.cs.washington.edu【6】 Detecting Syllable-Level Pronunciation Stress with A Self-Attention Model标题：用自我注意模型检测音节水平的发音重音链接：https://arxiv.org/abs/2311.00301作者：Wang Weiying,Nakajima Akinori备注：source codes available at this https URL摘要：有效的口语交流的一个先决条件是单词应该发音清楚，特别是对于非母语人士。词的重音是使英语清晰和正确的关键，音节重音的错位可能导致误解。因此，了解重音水平对英语使用者和学习者都很重要。本文提出了一个自我注意模型来识别英语口语中每个音节的重音水平。本文探讨了音节及其核心（音节的元音）的音高、强度、音长和类型等各种韵律和范畴特征。这些特征被输入到自我注意力模型，并预测音节级重音。最简单的模型在不同数据集上的准确率超过88%和93%，而更高级的模型提供更高的准确率。我们的研究表明，自我注意模型可以在压力水平检测的前景。这些模型可以应用于各种场景，如在线会议和英语学习。摘要：One precondition of effective oral communication is that words should be pronounced clearly, especially for non-native speakers. Word stress is the key to clear and correct English, and misplacement of syllable stress may lead to misunderstandings. Thus, knowing the stress level is important for English speakers and learners. This paper presents a self-attention model to identify the stress level for each syllable of spoken English. Various prosodic and categorical features, including the pitch level, intensity, duration and type of the syllable and its nuclei (the vowel of the syllable), are explored. These features are input to the self-attention model, and syllable-level stresses are predicted. The simplest model yields an accuracy of over 88% and 93% on different datasets, while more advanced models provide higher accuracy. Our study suggests that the self-attention model can be promising in stress-level detection. These models could be applied to various scenarios, such as online meetings and English learning.【7】 C2C: Cough to COVID-19 Detection in BHI 2023 Data Challenge标题：C2C：在BHI 2023数据挑战赛中检测到新冠肺炎咳嗽链接：https://arxiv.org/abs/2311.00364作者：Woo-Jin Chung,Miseul Kim,Hong-Goo Kang备注：1st place winning paper from the BHI 2023 Data Challenge Competition: Sensor Informatics摘要：本报告描述了我们提交给BHI 2023数据竞赛：传感器挑战的信息。我们的Audio Alchemists团队设计了一个基于声学的COVID-19诊断系统，咳嗽到COVID-19（C2C），并赢得了挑战赛的第一名。C2C涉及三个关键贡献：输入信号的预处理，利用Wav2vec2.0的咳嗽相关表示提取和数据增强。通过实验发现，我们证明了C2C通过咳嗽信号提高COVID-19诊断准确性的潜力。我们提出的模型在COVID-19诊断的背景下实现了0.7810的ROC-AUC值。实现细节和Python代码可以在以下链接中找到：https://github.com/Woo-jin-Chung/BHI_2023_challenge_Audio_Alchemists摘要：This report describes our submission to BHI 2023 Data Competition: Sensor challenge. Our Audio Alchemists team designed an acoustic-based COVID-19 diagnosis system, Cough to COVID-19 (C2C), and won the 1st place in the challenge. C2C involves three key contributions: pre-processing of input signals, cough-related representation extraction leveraging Wav2vec2.0, and data augmentation. Through experimental findings, we demonstrate C2C's promising potential to enhance the diagnostic accuracy of COVID-19 via cough signals. Our proposed model achieves a ROC-AUC value of 0.7810 in the context of COVID-19 diagnosis. The implementation details and the python code can be found in the following link: https://github.com/Woo-jin-Chung/BHI_2023_challenge_Audio_Alchemistseess.AS音频处理【1】 Reverberant sound field equalisation for an enhanced stereo playback experience标题：混响声场均衡，增强立体声播放体验链接：https://arxiv.org/abs/2311.00624作者：James Brooks-Park,Steven van de Par摘要：多年来，房间均衡一直是研究和产品开发的前沿课题，其目的是提高混响室中扬声器的播放质量。传统的房间均衡系统包括多个滤波器，当应用于主扬声器时，补偿额外的房间色彩。该出版物介绍了一种新颖的均衡技术，其中，通过两个环绕扬声器将伽马通滤波器频带能量添加到混响声场，使得来自主扬声器的直达声不变，但是直达和混响能量的总和在收听位置处被均衡。与传统系统不同，该方法允许直达声的目标函数与混响声场不同。所提出的方法的动机是直接和混响的声音成分在人类感知声音中发挥不同的作用。随着介绍所提出的方法，从主观听力测试的结果，展示了对所提出的技术相比，传统的房间均衡技术和立体声播放的偏好。摘要：The topic of room equalisation has been at the forefront of research and product development for many years, with the aim of increasing the playback quality of loudspeakers in reverberant rooms. Traditional room equalisation systems comprise of a number of filters that when applied to the primary loudspeakers, additional room colouration is compensated for. This publication introduces a novel equalisation technique where gammatone filter band energy is added to the reverberant sound field via two surround loudspeakers, leaving the direct sound from the primary loudspeakers unaltered, but the sum of direct and reverberant energy is equalised at the listening position. Unlike traditional systems, this method allows the target function of the direct sound to differ from the reverberant sound field. The proposed method is motivated by the different roles direct and reverberant sound components play in humans perception of sound. Along with introducing the proposed method, results from a subjective listening test are presented, demonstrating the preference towards the proposed technique when compared to a traditional room equalisation technique and stereo playback.【2】 An analysis of large speech models-based representations for speech emotion recognition标题：基于大规模语音模型的语音情感识别表示方法分析链接：https://arxiv.org/abs/2311.00394作者：Adrian Bogdan Stânea,Vlad Striletchi,Cosmin Striletchi,Adriana Stan备注：Presented at SPED2023 - IEEE 12th Conference on Speech Technology and Human-Computer Dialogue, october 2023, Bucharest, Romania摘要：大型语音模型衍生的功能最近在多个下游任务中表现出比基于信号的功能更高的性能，即使网络没有向目标任务进行微调。在本文中，我们展示了几个信号和神经模型衍生的语音情感识别功能的分析结果。我们使用预先训练的模型，并探索它们内在的潜在情感抽象。使用简单的分类方法，以便不干扰或增加知识的任务。我们表明，即使没有微调，这些大型神经语音模型的表示中的一些可以包含信息，使性能接近，甚至超过六个标准语音情感识别数据集的最先进的结果。摘要：Large speech models-derived features have recently shown increased performance over signal-based features across multiple downstream tasks, even when the networks are not finetuned towards the target task. In this paper we show the results of an analysis of several signal- and neural models-derived features for speech emotion recognition. We use pretrained models and explore their inherent potential abstractions of emotions. Simple classification methods are used so as to not interfere or add knowledge to the task. We show that, even without finetuning, some of these large neural speech models' representations can enclose information that enables performances close to, and even beyond state-of-the-art results across six standard speech emotion recognition datasets.【3】 C2C: Cough to COVID-19 Detection in BHI 2023 Data Challenge标题：C2C：在BHI 2023数据挑战赛中检测到新冠肺炎咳嗽链接：https://arxiv.org/abs/2311.00364作者：Woo-Jin Chung,Miseul Kim,Hong-Goo Kang备注：1st place winning paper from the BHI 2023 Data Challenge Competition: Sensor Informatics摘要：本报告描述了我们提交给BHI 2023数据竞赛：传感器挑战的信息。我们的Audio Alchemists团队设计了一个基于声学的COVID-19诊断系统，咳嗽到COVID-19（C2C），并赢得了挑战赛的第一名。C2C涉及三个关键贡献：输入信号的预处理，利用Wav2vec2.0的咳嗽相关表示提取和数据增强。通过实验发现，我们证明了C2C通过咳嗽信号提高COVID-19诊断准确性的潜力。我们提出的模型在COVID-19诊断的背景下实现了0.7810的ROC-AUC值。实现细节和Python代码可以在以下链接中找到：https://github.com/Woo-jin-Chung/BHI_2023_challenge_Audio_Alchemists摘要：This report describes our submission to BHI 2023 Data Competition: Sensor challenge. Our Audio Alchemists team designed an acoustic-based COVID-19 diagnosis system, Cough to COVID-19 (C2C), and won the 1st place in the challenge. C2C involves three key contributions: pre-processing of input signals, cough-related representation extraction leveraging Wav2vec2.0, and data augmentation. Through experimental findings, we demonstrate C2C's promising potential to enhance the diagnostic accuracy of COVID-19 via cough signals. Our proposed model achieves a ROC-AUC value of 0.7810 in the context of COVID-19 diagnosis. The implementation details and the python code can be found in the following link: https://github.com/Woo-jin-Chung/BHI_2023_challenge_Audio_Alchemists【4】 RIR-SF: Room Impulse Response Based Spatial Feature for Multi-channel Multi-talker ASR标题：RIR-SF：基于房间脉冲响应的多通道多说话人ASR空间特征链接：https://arxiv.org/abs/2311.00146作者：Yiwen Shao,Shi-Xiong Zhang,Dong Yu摘要：多通道多说话者自动语音识别（ASR）在语音社区中提出了持续的挑战，特别是当面临显著的混响效应时。在这项研究中，我们介绍了一种新的方法，涉及重叠的语音信号与房间脉冲响应（RIR）对应的目标扬声器的传输到麦克风阵列的卷积。这种创新技术产生了一种新的空间特征，称为RIR-SF。通过与先前建立的最先进的三维空间特征的全面比较，理论分析和实验结果证实了我们提出的RIR-SF的优越性。我们证明，RIR-SF优于现有的方法，导致显着的21.3%的相对减少的字符错误率（CER）在多通道多说话人ASR系统。重要的是，这种新功能在面对强混响时表现出鲁棒性，超越了以前方法的局限性。摘要：Multi-channel multi-talker automatic speech recognition (ASR) presents ongoing challenges within the speech community, particularly when confronted with significant reverberation effects. In this study, we introduce a novel approach involving the convolution of overlapping speech signals with the room impulse response (RIR) corresponding to the target speaker's transmission to a microphone array. This innovative technique yields a novel spatial feature known as the RIR-SF. Through a comprehensive comparison with the previously established state-of-the-art 3D spatial feature, both theoretical analysis and experimental results substantiate the superiority of our proposed RIR-SF. We demonstrate that the RIR-SF outperforms existing methods, leading to a remarkable 21.3\% relative reduction in the Character Error Rate (CER) in multi-channel multi-talker ASR systems. Importantly, this novel feature exhibits robustness in the face of strong reverberation, surpassing the limitations of previous approaches.【5】 End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation标题：端到端单声道话轮感知会话语音翻译链接：https://arxiv.org/abs/2311.00697作者：Juan Zuluaga-Gomez,Zhaocheng Huang,Xing Niu,Rohit Paturi,Sundararajan Srinivasan,Prashant Mathur,Brian Thompson,Marcello Federico备注：Accepted at EMNLP 2023. Code: this https URL摘要：传统的语音到文本翻译（ST）系统是在单个说话者的话语上训练的，并且它们可能无法推广到音频包含多个说话者的对话的现实生活场景。在本文中，我们解决了单通道多扬声器会话ST的端到端和多任务的训练模型，命名为Speaker-Turn Aware对话语音翻译，它结合了自动语音识别，语音翻译和扬声器转向检测使用特殊的标记在一个序列化的标签格式。我们在Fisher-CALLHOME语料库上进行了实验，我们通过将两个单扬声器通道合并为一个多扬声器通道进行了调整，从而代表了具有多扬声器转弯和串扰的更现实和更具挑战性的场景。在单扬声器和多扬声器条件下以及针对传统ST系统的实验结果表明，我们的模型在多扬声器条件下的性能优于参考系统，同时在单扬声器条件下达到相当的性能。我们发布用于数据处理和模型训练的脚本。摘要：Conventional speech-to-text translation (ST) systems are trained on single-speaker utterances, and they may not generalize to real-life scenarios where the audio contains conversations by multiple speakers. In this paper, we tackle single-channel multi-speaker conversational ST with an end-to-end and multi-task training model, named Speaker-Turn Aware Conversational Speech Translation, that combines automatic speech recognition, speech translation and speaker turn detection using special tokens in a serialized labeling format. We run experiments on the Fisher-CALLHOME corpus, which we adapted by merging the two single-speaker channels into one multi-speaker channel, thus representing the more realistic and challenging scenario with multi-speaker turns and cross-talk. Experimental results across single- and multi-speaker conditions and against conventional ST systems, show that our model outperforms the reference systems on the multi-speaker condition, while attaining comparable performance on the single-speaker condition. We release scripts for data processing and model training.【6】 Controllable Music Production with Diffusion Models and Guidance Gradients标题：具有扩散模型和引导梯度的可控音乐制作链接：https://arxiv.org/abs/2311.00613作者：Mark Levy,Bruno Di Giorgi,Floris Weers,Angelos Katharopoulos,Tom Nickson摘要：我们演示了如何从扩散模型的条件生成可以用来解决各种现实的任务，在44.1kHz的立体声音频与采样时间的指导下的音乐生产。我们考虑的场景包括音乐音频的延续，修复和再生，在两个不同的音乐曲目之间创建平滑过渡，以及将所需的风格特征转移到现有的音频片段。我们通过在采样时在一个简单的框架中应用指导来实现这一点，该框架支持重建和分类损失，或两者的任何组合。这种方法确保生成的音频可以匹配其周围的上下文，或者符合相对于任何合适的预先训练的分类器或嵌入模型指定的类分布或潜在表示。摘要：We demonstrate how conditional generation from diffusion models can be used to tackle a variety of realistic tasks in the production of music in 44.1kHz stereo audio with sampling-time guidance. The scenarios we consider include continuation, inpainting and regeneration of musical audio, the creation of smooth transitions between two different music tracks, and the transfer of desired stylistic characteristics to existing audio clips. We achieve this by applying guidance at sampling time in a simple framework that supports both reconstruction and classification losses, or any combination of the two. This approach ensures that generated audio can match its surrounding context, or conform to a class distribution or latent representation specified relative to any suitable pre-trained classifier or embedding model.【7】 Active Noise Control Portable Device Design标题：有源噪声控制便携式装置的设计链接：https://arxiv.org/abs/2311.00535作者：kai Wu,Yuanyuan Chen摘要：虽然我们的世界充满了我们无法抗拒享受的自然声音，但它也充满了其他可能令人恼火的声音，这就是噪音。噪声不仅影响工作效率，而且影响人体健康。降低噪声是一个非常重要和困难的问题。多年来，这个问题一直在以多种方式解决。现有的降噪方法大多依赖于材料和传输介质，仅对高频噪声有一定的抑制作用。然而，有效的降噪方法，特别是对低频噪声的降噪方法非常有限。 在这里，我们提出了一个降噪系统，包括一个传感器来检测环境中的噪声。然后，噪声将被发送到电子控制系统以处理噪声，电子控制系统将产生反相频率信号以抵消干扰。最后，处理后的较小噪声将由扬声器广播。通过这种智能降噪系统，即使是低频噪音也可以消除。 该系统还集成了睡眠跟踪和音乐播放器应用程序。它还可以记忆和存储相同环境的设置，感知温度，以及智能控制家居家具，火灾报警等，这个智能系统可以通过Wi-Fi或蓝牙轻松传输数据并通过其APP进行控制。 在本项目中，我们将展示上述技术的一个模型，该模型可用于各种环境，以防止噪音污染，并为难以找到安静的睡眠，工作或学习环境的人提供解决方案。摘要：While our world is filled with its own natural sounds that we can't resist enjoying, it is also chock-full of other sounds that can be irritating, this is noise. Noise not only influences the working efficiency but also the human's health. The problem of reducing noise is one of great importance and great difficulty. The problem has been addressed in many ways over the years. The current methods for noise reducing mostly rely on the materials and transmission medium, which are only effective to some extent for the high frequency noise. However, the effective reduction noise method especially for low frequency noise is very limited. Here we come up with a noise reduction system consist of a sensor to detect the noise in the environment. Then the noise will be sent to an electronic control system to process the noise, which will generate a reverse phase frequency signal to counteract the disturbance. Finally, the processed smaller noise will be broadcasted by the speaker. Through this smart noise reduction system, even the noise with low-frequency can be eliminated. The system is also integrated with sleep tracking and music player applications. It can also remember and store settings for the same environment, sense temperature, and smart control of home furniture, fire alarm, etc. This smart system can transfer data easily by Wi-Fi or Bluetooth and controlled by its APP. In this project, we will present a model of the above technology which can be used in various environments to prevent noise pollution and provide a solution to the people who have difficulties finding a peaceful and quiet environment for sleep, work or study.【8】 Deep Neural Networks for Automatic Speaker Recognition Do Not Learn Supra-Segmental Temporal Features标题：用于说话人自动识别的深度神经网络不学习超音段时间特征链接：https://arxiv.org/abs/2311.00489作者：Daniel Neururer,Volker Dellow,Thilo Stadelmann备注：7 pages, 2 figures, 3 tables摘要：虽然深度神经网络在自动说话人识别和相关任务中显示出令人印象深刻的结果，但令人不满的是，人们对这些结果的确切原因知之甚少。在先前的工作中，部分成功归因于他们对超节段时间信息（SST）建模的能力，即，除了频谱特征之外，还学习语音节奏韵律特征。在本文中，我们（i）提出并应用一种新的测试来量化最先进的说话人识别神经网络的性能在多大程度上可以通过对SST建模来解释;（ii）提出几种方法来迫使各自的网络更多地关注SST并评估其优点。我们发现，各种基于CNN和RNN的神经网络架构的说话人识别没有模型SST到任何足够的程度，即使是被迫的。这些结果为未来更好地利用完整语音信号的有影响力的研究提供了高度相关的基础，并深入了解了此类网络的内部工作原理，增强了语音技术深度学习的可解释性。摘要：While deep neural networks have shown impressive results in automatic speaker recognition and related tasks, it is dissatisfactory how little is understood about what exactly is responsible for these results. Part of the success has been attributed in prior work to their capability to model supra-segmental temporal information (SST), i.e., learn rhythmic-prosodic characteristics of speech in addition to spectral features. In this paper, we (i) present and apply a novel test to quantify to what extent the performance of state-of-the-art neural networks for speaker recognition can be explained by modeling SST; and (ii) present several means to force respective nets to focus more on SST and evaluate their merits. We find that a variety of CNN- and RNN-based neural network architectures for speaker recognition do not model SST to any sufficient degree, even when forced. The results provide a highly relevant basis for impactful future research into better exploitation of the full speech signal and give insights into the inner workings of such networks, enhancing explainability of deep learning for speech technologies.【9】 Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling标题：Distil-Whisper：基于大规模伪标注的稳健知识提取链接：https://arxiv.org/abs/2311.00430作者：Sanchit Gandhi,Patrick von Platen,Alexander M. Rush备注：30 pages, 2 figures, 25 tables摘要：随着预先训练的语音识别模型的大小增加，在低延迟或资源受限的环境中运行这些大型模型变得具有挑战性。在这项工作中，我们利用伪标签来组装一个大规模的开源数据集，我们使用它来将Whisper模型提取成一个较小的变体，称为Distil-Whisper。使用简单的单词错误率（WER）启发式算法，我们只选择最高质量的伪标签进行训练。蒸馏模型的速度提高了5.8倍，参数减少了51%，同时在zero-shot传输设置中对分布外测试数据的WER在1%以内。Distil-Whisper保持了Whisper模型对困难声学条件的鲁棒性，同时在长格式音频上不太容易出现幻觉错误。Distil-Whisper旨在与Whisper配对进行推测性解码，产生2倍的加速，同时在数学上确保与原始模型相同的输出。为了促进这一领域的进一步研究，我们公开了我们的训练代码、推理代码和模型。摘要：As the size of pre-trained speech recognition models increases, running these large models in low-latency or resource-constrained environments becomes challenging. In this work, we leverage pseudo-labelling to assemble a large-scale open-source dataset which we use to distill the Whisper model into a smaller variant, called Distil-Whisper. Using a simple word error rate (WER) heuristic, we select only the highest quality pseudo-labels for training. The distilled model is 5.8 times faster with 51% fewer parameters, while performing to within 1% WER on out-of-distribution test data in a zero-shot transfer setting. Distil-Whisper maintains the robustness of the Whisper model to difficult acoustic conditions, while being less prone to hallucination errors on long-form audio. Distil-Whisper is designed to be paired with Whisper for speculative decoding, yielding a 2 times speed-up while mathematically ensuring the same outputs as the original model. To facilitate further research in this domain, we make our training code, inference code and models publicly accessible.【10】 Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables标题：语义听觉：用双耳可听物编程声学场景链接：https://arxiv.org/abs/2311.00320作者：Bandhav Veluri,Malek Itani,Justin Chan,Takuya Yoshioka,Shyamnath Gollakota摘要：想象一下，能够在公园里听鸟儿啁啾，而不会听到其他徒步旅行者的喋喋不休，或者能够在繁忙的街道上屏蔽交通噪音，同时仍然能够听到紧急警报器和汽车喇叭声。我们引入语义听觉，这是一种新的听觉设备功能，使它们能够实时关注或忽略来自现实环境的特定声音，同时还保留了空间线索。为了实现这一目标，我们做出了两项技术贡献：1）我们提出了第一个神经网络，可以在存在干扰声音和背景噪声的情况下实现双耳目标声音提取，2）我们设计了一种训练方法，使我们的系统能够推广到现实世界中使用。结果表明，我们的系统可以运行20个声音类，我们基于变压器的网络在连接的智能手机上的运行时间为6.56 ms。在以前看不见的室内和室外场景中对参与者进行的野外评估表明，我们的概念验证系统可以提取目标声音并概括以保留其双耳输出中的空间线索。项目页面代码：https://semantichearing.cs.washington.edu摘要：Imagine being able to listen to the birds chirping in a park without hearing the chatter from other hikers, or being able to block out traffic noise on a busy street while still being able to hear emergency sirens and car honks. We introduce semantic hearing, a novel capability for hearable devices that enables them to, in real-time, focus on, or ignore, specific sounds from real-world environments, while also preserving the spatial cues. To achieve this, we make two technical contributions: 1) we present the first neural network that can achieve binaural target sound extraction in the presence of interfering sounds and background noise, and 2) we design a training methodology that allows our system to generalize to real-world use. Results show that our system can operate with 20 sound classes and that our transformer-based network has a runtime of 6.56 ms on a connected smartphone. In-the-wild evaluation with participants in previously unseen indoor and outdoor scenarios shows that our proof-of-concept system can extract the target sounds and generalize to preserve the spatial cues in its binaural output. Project page with code: https://semantichearing.cs.washington.edu【11】 Detecting Syllable-Level Pronunciation Stress with A Self-Attention Model标题：用自我注意模型检测音节水平的发音重音链接：https://arxiv.org/abs/2311.00301作者：Wang Weiying,Nakajima Akinori备注：source codes available at this https URL摘要：有效的口语交流的一个先决条件是单词应该发音清楚，特别是对于非母语人士。词的重音是使英语清晰和正确的关键，音节重音的错位可能导致误解。因此，了解重音水平对英语使用者和学习者都很重要。本文提出了一个自我注意模型来识别英语口语中每个音节的重音水平。本文探讨了音节及其核心（音节的元音）的音高、强度、音长和类型等各种韵律和范畴特征。这些特征被输入到自我注意力模型，并预测音节级重音。最简单的模型在不同数据集上的准确率超过88%和93%，而更高级的模型提供更高的准确率。我们的研究表明，自我注意模型可以在压力水平检测的前景。这些模型可以应用于各种场景，如在线会议和英语学习。摘要：One precondition of effective oral communication is that words should be pronounced clearly, especially for non-native speakers. Word stress is the key to clear and correct English, and misplacement of syllable stress may lead to misunderstandings. Thus, knowing the stress level is important for English speakers and learners. This paper presents a self-attention model to identify the stress level for each syllable of spoken English. Various prosodic and categorical features, including the pitch level, intensity, duration and type of the syllable and its nuclei (the vowel of the syllable), are explored. These features are input to the self-attention model, and syllable-level stresses are predicted. The simplest model yields an accuracy of over 88% and 93% on different datasets, while more advanced models provide higher accuracy. Our study suggests that the self-attention model can be promising in stress-level detection. These models could be applied to various scenarios, such as online meetings and English learning.
每日语音速递
