今日论文合集：cs.SD语音11篇，eess.AS音频处理15篇。
cs.SD语音
【1】 Emotion-Conditioned Melody Harmonization with Hierarchical Variational  Autoencoder标题：基于分层变分自动编码器的情感条件旋律和声链接：https://arxiv.org/abs/2306.03718
作者：Shulei Ji,Xinyu Yang机构：School of computer science and technology, Xi’an Jiaotong University, Xi’an, China备注：Accepted by IEEE SMC 2023摘要：现有的旋律和声模型在提高生成和声的质量方面取得了很大的进展，但大多数都忽略了音乐背后的情感。同时，以前的方法产生的谐波的可变性是不够的。为了解决这些问题，我们提出了一种新的基于LSTM的分层变分自动编码器（LHVAE）调查的情绪条件对旋律和谐的影响，同时提高生成的和谐的质量，并捕捉丰富的变化和弦进行。具体而言，LHVAE在不同的级别（块和酒吧级），以模拟全球和本地的音乐属性的潜在变量和情绪条件。此外，我们在每一步引入了一个基于注意力的旋律上下文向量，以更好地学习旋律和和声之间的对应关系。客观评价的实验结果表明，我们提出的模型优于其他基于LSTM的模型。通过主观评价，我们得出结论，仅仅改变和弦几乎没有改变音乐的整体情感。定性分析表明，我们的模型产生变量谐波的能力。摘要：Existing melody harmonization models have made great progress in improving the quality of generated harmonies, but most of them ignored the emotions beneath the music. Meanwhile, the variability of harmonies generated by previous methods is insufficient. To solve these problems, we propose a novel LSTM-based Hierarchical Variational Auto-Encoder (LHVAE) to investigate the influence of emotional conditions on melody harmonization, while improving the quality of generated harmonies and capturing the abundant variability of chord progressions. Specifically, LHVAE incorporates latent variables and emotional conditions at different levels (piece- and bar-level) to model the global and local music properties. Additionally, we introduce an attention-based melody context vector at each step to better learn the correspondence between melodies and harmonies. Experimental results of the objective evaluation show that our proposed model outperforms other LSTM-based models. Through subjective evaluation, we conclude that only altering the chords hardly changes the overall emotion of the music. The qualitative analysis demonstrates the ability of our model to generate variable harmonies.
【2】 Dance Generation by Sound Symbolic Words标题：有声象征性词语的舞蹈生成链接：https://arxiv.org/abs/2306.03646
作者：Miki Okamura,Naruya Kondo,Tatsuki Fushimi Maki Sakamoto,Yoichi Ochiai机构：TATSUKI FUSHIMI, University of Tsukuba, Japan, MAKI SAKAMOTO, The University of Electro-Communications, Japan摘要：本研究提出一种以拟声词为输入的舞蹈动作生成方法，旨在提高舞蹈动作生成的创造性和多样性。与文字和音乐不同，拟声词通过抽象的词语表达来传达节奏和意义，不受表达的限制，也不需要专门的知识。我们采用AI编舞框架，并采用Sakamoto系统，这是一种专注于音素和音节的拟声词特征提取方法。此外，我们提出了一个新的数据集的40拟声舞蹈运动对通过用户调查收集。我们的研究结果表明，所提出的方法，使更直观的舞蹈生成，并可以创建舞蹈动作，使用声音符号的话，从各种语言，包括那些没有象声词。这突出了跨越不同语言和文化的多样化舞蹈创作的潜力，可供更广泛的观众使用。来自我们模型的定性样本可以在以下位置找到：https: sites.google.com view onomatopoeia-dance home 。摘要：This study introduces a novel approach to generate dance motions using onomatopoeia as input, with the aim of enhancing creativity and diversity in dance generation. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions without constraints on expression and without need for specialized knowledge. We adapt the AI Choreographer framework and employ the Sakamoto system, a feature extraction method for onomatopoeia focusing on phonemes and syllables. Additionally, we present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey. Our results demonstrate that the proposed method enables more intuitive dance generation and can create dance motions using sound-symbolic words from a variety of languages, including those without onomatopoeia. This highlights the potential for diverse dance creation across different languages and cultures, accessible to a wider audience. Qualitative samples from our model can be found at: https: sites.google.com view onomatopoeia-dance home .
【3】 Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis标题：Ada-TTA：面向自适应的高质量文本到对话化身合成链接：https://arxiv.org/abs/2306.03504
作者：Zhenhui Ye,Ziyue Jiang,Yi Ren,Jinglin Liu,Chen Zhang,Xiang Yin,Zejun Ma,Zhou Zhao备注：6 pages, 3 figures摘要：我们感兴趣的是一个新的任务，即低资源的文本到说话的化身。给定只有几分钟长的说话人视频与音轨作为训练数据和任意文本作为驱动输入，我们的目标是合成高质量的说话的人像视频对应的输入文本。该任务在数字人行业具有广阔的应用前景，但由于两个挑战，在技术上尚未实现：（1）对于传统的多说话人文本到语音系统，模仿来自域外音频的音色是具有挑战性的。(2)在训练数据有限的情况下，很难呈现高保真和嘴唇同步的说话化身。本文介绍了自适应文本到说话化身（Ada-TTA），它（1）设计了一个通用的zero-shot多说话人TTS模型，能够很好地分离文本内容、音色和韵律;以及（2）包含神经渲染的最新进展以实现逼真的音频驱动的说话面部视频生成。通过这些设计，我们的方法克服了上述两个挑战，并实现了生成身份保留的语音和真实的说话人视频。实验表明，我们的方法可以合成逼真的，身份保护，视听同步说话的化身视频。摘要：We are interested in a novel task, namely low-resource text-to-talking avatar. Given only a few-minute-long talking person video with the audio track as the training data and arbitrary texts as the driving input, we aim to synthesize high-quality talking portrait videos corresponding to the input text. This task has broad application prospects in the digital human industry but has not been technically achieved yet due to two challenges: (1) It is challenging to mimic the timbre from out-of-domain audio for a traditional multi-speaker Text-to-Speech system. (2) It is hard to render high-fidelity and lip-synchronized talking avatars with limited training data. In this paper, we introduce Adaptive Text-to-Talking Avatar (Ada-TTA), which (1) designs a generic zero-shot multi-speaker TTS model that well disentangles the text content, timbre, and prosody; and (2) embraces recent advances in neural rendering to achieve realistic audio-driven talking face video generation. With these designs, our method overcomes the aforementioned two challenges and achieves to generate identity-preserving speech and realistic talking person video. Experiments demonstrate that our method could synthesize realistic, identity-preserving, and audio-visual synchronized talking avatar videos.
【4】 Automatic Assessment of Oral Reading Accuracy for Reading Diagnostics标题：阅读诊断中口语阅读准确度的自动评估链接：https://arxiv.org/abs/2306.03444
作者：Bo Molenaar,Cristian Tejedor-Garcia,Helmer Strik,Catia Cucchiarini机构：Centre for Language and Speech Technology, Radboud University Nijmegen, Netherlands摘要：利用自动语音识别（ASR）技术对阅读流畅性进行自动评估，对于早期发现阅读困难和及时干预具有巨大的潜力。需要精确的评估工具，尤其是英语以外的语言。在这项研究中，我们评估了六个国家的最先进的ASR为基础的系统，自动评估荷兰语口语阅读的准确性使用Kaldi和Whisper。结果表明，我们最成功的系统与人类评价达成了实质性协议（MCC = .63）。同一系统达到了强制解码置信度得分和单词正确性之间的最高相关性（r = .45）。该系统的语言模型（LM）由人工正字法转录和测试数据的阅读提示组成，这表明在LM中包括阅读错误提高了评估性能。我们讨论了发展自动评估系统的影响，并确定了未来研究的可能途径。摘要：Automatic assessment of reading fluency using automatic speech recognition (ASR) holds great potential for early detection of reading difficulties and subsequent timely intervention. Precise assessment tools are required, especially for languages other than English. In this study, we evaluate six state-of-the-art ASR-based systems for automatically assessing Dutch oral reading accuracy using Kaldi and Whisper. Results show our most successful system reached substantial agreement with human evaluations (MCC = .63). The same system reached the highest correlation between forced decoding confidence scores and word correctness (r = .45). This system's language model (LM) consisted of manual orthographic transcriptions and reading prompts of the test data, which shows that including reading errors in the LM improves assessment performance. We discuss the implications for developing automatic assessment systems and identify possible avenues of future research.
【5】 Alzheimer Disease Classification through ASR-based Transcriptions:  Exploring the Impact of Punctuation and Pauses标题：通过基于ASR的转录进行阿尔茨海默病分类：探索标点符号和停顿的影响链接：https://arxiv.org/abs/2306.03443
作者：Lucía Gómez-Zaragozá,Simone Wills,Cristian Tejedor-Garcia,Javier Marín-Morales,Mariano Alcañiz,Helmer Strik机构：Instituto Universitario de Investigacio´n en Tecnolog´ıa Centrada en el Ser Humano, Universitat, Politecnica de Valencia, Spain,  Radboud University, The Netherlands摘要：阿尔茨海默氏病（AD）是世界上主要的神经退行性疾病，其通常导致交流困难。分析语音可以作为诊断工具来识别病情。最近的ADReSS挑战为AD分类提供了数据集，并强调了手动转录的实用性。在这项研究中，我们使用了新的国家的最先进的自动语音识别（ASR）模型耳语获得的转录，其中还包括自动标点符号。分类模型分别在手动和ASR成绩单上实现了0.854和0.833的测试准确度分数，结合了预训练的FastText单词嵌入和递归神经网络。此外，我们探讨了包括暂停信息和标点符号的转录的影响。我们发现，标点符号在某些情况下只产生轻微的改善，而暂停编码辅助AD分类手动和ASR转录在所有的方法研究。摘要：Alzheimer's Disease (AD) is the world's leading neurodegenerative disease, which often results in communication difficulties. Analysing speech can serve as a diagnostic tool for identifying the condition. The recent ADReSS challenge provided a dataset for AD classification and highlighted the utility of manual transcriptions. In this study, we used the new state-of-the-art Automatic Speech Recognition (ASR) model Whisper to obtain the transcriptions, which also include automatic punctuation. The classification models achieved test accuracy scores of 0.854 and 0.833 combining the pretrained FastText word embeddings and recurrent neural networks on manual and ASR transcripts respectively. Additionally, we explored the influence of including pause information and punctuation in the transcriptions. We found that punctuation only yielded minor improvements in some cases, whereas pause encoding aided AD classification for both manual and ASR transcriptions across all approaches investigated.
【6】 Phase perturbation improves channel robustness for speech spoofing  countermeasures标题：相位扰动提高语音欺骗对抗的信道稳健性链接：https://arxiv.org/abs/2306.03389
作者：Yongyi Zang,You Zhang,Zhiyao Duan机构：University of Rochester, Rochester, New York, USA备注：5 pages; Accepted to INTERSPEECH 2023摘要：在本文中，我们的目标是解决语音对抗（CM）系统，这是用来区分合成语音从人类自然语音的信道鲁棒性的问题。在两个假设的基础上，我们提出了一种在时域CM系统的训练过程中扰动相位信息的方法。通信网络通常采用有损压缩编解码器，其仅对幅度信息进行编码，因此严重改变相位信息。此外，现有技术的CM系统依赖于相位信息来识别欺骗语音。因此，我们认为有损压缩编解码器引起的相位域中的信息损失降低了看不见的信道的性能。我们首先建立依赖的时域CM系统的相位信息的扰动相位评估，表现出强烈的退化。然后，我们证明了在训练过程中的扰动阶段导致显着的性能改善，而扰动幅度导致进一步退化。摘要：In this paper, we aim to address the problem of channel robustness in speech countermeasure (CM) systems, which are used to distinguish synthetic speech from human natural speech. On the basis of two hypotheses, we suggest an approach for perturbing phase information during the training of time-domain CM systems. Communication networks often employ lossy compression codec that encodes only magnitude information, therefore heavily altering phase information. Also, state-of-the-art CM systems rely on phase information to identify spoofed speech. Thus, we believe the information loss in the phase domain induced by lossy compression codec degrades the performance of the unseen channel. We first establish the dependence of time-domain CM systems on phase information by perturbing phase in evaluation, showing strong degradation. Then, we demonstrated that perturbing phase during training leads to a significant performance improvement, whereas perturbing magnitude leads to further degradation.
【7】 Reef Elegy: An Auditory Display of Hawaii's 2019 Coral Bleaching Data标题：珊瑚礁悲歌：夏威夷2019年珊瑚白化数据的听觉展示链接：https://arxiv.org/abs/2306.03307
作者：Stefano Kalonaris机构：Music Information Intelligence Team, Center for Advanced Intelligence Project (AIP), RIKEN, Japan备注：To appear in: Proceedings of the 28th International Conference on Auditory Display (ICAD 2023)摘要：本文通过空间音频和参数映射方法描述了夏威夷2019年珊瑚白化数据的听觉显示。选定的数据字段跨越78天被映射到珊瑚礁的自然音景，这是逐步改变其组成元素的相应的珊瑚位置进行漂白的声音代理。对于这些元素中的一些，该过程概述了从密集到稀疏的、减少的音景的轨迹，而对于其他元素，该过程从谐波音调向复杂频谱转变。这个实验是伴随着一个简短的评价研究，它在一个既定的审美视角空间的背景下，并探讨其潜在的公众参与的话语围绕气候变化。摘要：This paper describes an auditory display of Hawaii's 2019 coral bleaching data via means of spatial audio and parameter mapping methods. Selected data fields spanning 78 days are mapped to sound surrogates of coral reefs' natural soundscapes, which are progressively altered in their constituent elements as the corresponding coral locations undergo bleaching. For some of these elements, this process outlines a trajectory from a dense to a sparser, reduced soundscape, while for others it translates moving away from harmonic tones and towards complex spectra. This experiment is accompanied by a short evaluation study to contextualize it in an established aesthetic perspective space and to probe its potential for public engagement in the discourse around climate change.
【8】 DeepVQE: Real Time Deep Voice Quality Enhancement for Joint Acoustic  Echo Cancellation, Noise Suppression and Dereverberation标题：DeepVQE：用于联合声学回波消除、噪声抑制和去混响的实时深音质量增强链接：https://arxiv.org/abs/2306.03177
作者：Evgenii Indenbom,Nicolae-Catalin Ristea,Ando Saabas,Tanel Parnamaa,Jegor Guzvin,Ross Cutler机构：Cutler, Microsoft Corp.摘要：声学回声消除（AEC）、噪声抑制（NS）和去混响（DR）是现代全双工通信系统的组成部分。随着对电话会议系统的需求增加，需要解决这些任务以获得有效且高效的在线会议体验。大多数先前的研究分别提出了这些任务的解决方案，将它们与基于数字信号处理（DSP）的组件相结合，导致复杂的管道，这些管道通常在现实世界的应用中部署是不切实际的。本文提出了一种名为DeepVQE的实时交叉注意深度模型，该模型基于残差卷积神经网络（CNN）和递归神经网络（RNN），以同时解决AEC、NS和DR。我们进行了几次消融研究，以分析我们的模型的不同组成部分的整体性能的贡献。DeepVQE在ICASSP 2023声学回声消除挑战赛和ICASSP 2023深度噪声抑制挑战赛测试集的非个性化曲目上实现了最先进的性能，表明单个模型可以以出色的性能处理多个任务。此外，该模型实时运行，并已成功地测试了微软团队平台。摘要：Acoustic echo cancellation (AEC), noise suppression (NS) and dereverberation (DR) are an integral part of modern full-duplex communication systems. As the demand for teleconferencing systems increases, addressing these tasks is required for an effective and efficient online meeting experience. Most prior research proposes solutions for these tasks separately, combining them with digital signal processing (DSP) based components, resulting in complex pipelines that are often impractical to deploy in real-world applications. This paper proposes a real-time cross-attention deep model, named DeepVQE, based on residual convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to simultaneously address AEC, NS, and DR. We conduct several ablation studies to analyze the contributions of different components of our model to the overall performance. DeepVQE achieves state-of-the-art performance on non-personalized tracks from the ICASSP 2023 Acoustic Echo Cancellation Challenge and ICASSP 2023 Deep Noise Suppression Challenge test sets, showing that a single model can handle multiple tasks with excellent performance. Moreover, the model runs in real-time and has been successfully tested for the Microsoft Teams platform.
【9】 Some voices are too common: Building fair speech recognition systems  using the Common Voice dataset标题：有些声音太常见了：使用通用声音数据集构建公平的语音识别系统链接：https://arxiv.org/abs/2306.03773
作者：Lucas Maison,Yannick Estève机构：Laboratoire Informatique d’Avignon, Avignon, France,  Thales SIX, Multimedia Lab, Gennevilliers, France备注：5 pages, 3 figures. Accepted to Interspeech 2023摘要：自动语音识别（ASR）系统变得越来越高效，这要归功于神经网络训练的新进展，如自监督学习。然而，他们被认为是不公平的某些群体，例如，人们说话的口音。在这项工作中，我们使用法国普通语音数据集来量化预训练的wav 2 vec ~2.0模型对几个人口统计群体的偏差。通过在各种固定大小、精心制作的训练集上对预训练模型进行微调，我们证明了说话人多样性的重要性。我们还对通用语音语料库进行了深入分析，并确定了该数据集用户应考虑的重要缺点。摘要：Automatic speech recognition (ASR) systems become increasingly efficient thanks to new advances in neural network training like self-supervised learning. However, they are known to be unfair toward certain groups, for instance, people speaking with an accent. In this work, we use the French Common Voice dataset to quantify the biases of a pre-trained wav2vec~2.0 model toward several demographic groups. By fine-tuning the pre-trained model on a variety of fixed-size, carefully crafted training sets, we demonstrate the importance of speaker diversity. We also run an in-depth analysis of the Common Voice corpus and identify important shortcomings that should be taken into account by users of this dataset.
【10】 Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive  Bias标题：Mega-TTS：具有内在感应偏差的大规模零命中率文本到语音转换链接：https://arxiv.org/abs/2306.03509
作者：Ziyue Jiang,Yi Ren,Zhenhui Ye,Jinglin Liu,Chen Zhang,Qian Yang,Shengpeng Ji,Rongjie Huang,Chunfeng Wang,Xiang Yin,Zejun Ma,Zhou Zhao机构：♠Zhejiang University & ♡ByteDance摘要：将文本到语音缩放到大型和野生数据集已被证明在实现音色和语音风格泛化方面非常有效，特别是在zero-shot TTS中。然而，以往的工作通常使用音频编解码器将语音编码成潜在的，并使用自回归语言模型或扩散模型来生成它，这忽略了语音的内在本质，并可能导致劣质或不可控的结果。我们认为语音可以分解为几个属性（例如，内容、音色、韵律和相位），并且它们中的每一个都应当使用具有适当归纳偏差的模块来建模。从这个角度出发，我们精心设计了一个新颖的大型zero-shot TTS系统，称为Mega-TTS，它使用大规模野生数据进行训练，并以不同的方式对不同的属性进行建模：1）代替使用由音频编解码器编码的潜在特征作为中间特征，我们仍然选择频谱图，因为它很好地分离了相位和其他属性。相位可以由基于GAN的声码器适当地构造，并且不需要由语言模型建模。2)我们使用全局向量对音色进行建模，因为音色是随时间缓慢变化的全局属性。3)我们进一步使用基于VQGAN的声学模型来生成声谱图和潜在代码语言模型来拟合韵律的分布，因为韵律在句子中随着时间的推移而快速变化，并且语言模型可以捕获本地和远程依赖关系。我们将Mega-TTS扩展到具有20 K小时语音的多域数据集，并评估其在看不见的扬声器上的性能。实验结果表明，Mega-TTS超越国家的最先进的TTS系统的zero-shot TTS，语音编辑，跨语言的TTS任务，具有优越的自然性，鲁棒性，和扬声器相似性，由于适当的归纳偏见的每个模块。音频样本可在www.example.com上获得https: mega-tts.github.io demo-page。摘要：Scaling text-to-speech to a large and wild dataset has been proven to be highly effective in achieving timbre and speech style generalization, particularly in zero-shot TTS. However, previous works usually encode speech into latent using audio codec and use autoregressive language models or diffusion models to generate it, which ignores the intrinsic nature of speech and may lead to inferior or uncontrollable results. We argue that speech can be decomposed into several attributes (e.g., content, timbre, prosody, and phase) and each of them should be modeled using a module with appropriate inductive biases. From this perspective, we carefully design a novel and large zero-shot TTS system called Mega-TTS, which is trained with large-scale wild data and models different attributes in different ways: 1) Instead of using latent encoded by audio codec as the intermediate feature, we still choose spectrogram as it separates the phase and other attributes very well. Phase can be appropriately constructed by the GAN-based vocoder and does not need to be modeled by the language model. 2) We model the timbre using global vectors since timbre is a global attribute that changes slowly over time. 3) We further use a VQGAN-based acoustic model to generate the spectrogram and a latent code language model to fit the distribution of prosody, since prosody changes quickly over time in a sentence, and language models can capture both local and long-range dependencies. We scale Mega-TTS to multi-domain datasets with 20K hours of speech and evaluate its performance on unseen speakers. Experimental results demonstrate that Mega-TTS surpasses state-of-the-art TTS systems on zero-shot TTS, speech editing, and cross-lingual TTS tasks, with superior naturalness, robustness, and speaker similarity due to the proper inductive bias of each module. Audio samples are available at https: mega-tts.github.io demo-page.
【11】 LipVoicer: Generating Speech from Silent Videos Guided by Lip Reading标题：LipVoier：由Lip Reading引导的无声视频生成语音链接：https://arxiv.org/abs/2306.03258
作者：Yochai Yemini,Aviv Shamsian,Lior Bracha,Sharon Gannot,Ethan Fetaya机构：Bar-Ilan University摘要：唇转语音涉及生成与人说话的无声视频同步的听起来自然的语音。尽管最近的进展，目前的方法仍然不能产生高质量的语音与高水平的可理解性的挑战性和现实的数据集，如LRS3。在这项工作中，我们提出了LipVoicer，一种新的方法，生成高质量的语音，即使在野外和丰富的数据集，通过将文本模态。给定一个无声视频，我们首先使用预先训练的唇读网络预测口语文本。然后，我们在视频上设置扩散模型，并通过分类器指导机制使用提取的文本，其中预先训练的ASR作为分类器。LipVoicer在LRS2和LRS3上的表现优于多个唇语基线，这是一个野外数据集，在其测试集中有数百个独特的说话者和不受限制的词汇。此外，我们的实验表明，包含文本模态在所产生的语音的可懂度中起着重要作用，在收听时容易察觉，并且根据经验反映在WER度量的大幅降低中。我们证明了LipVoicer的有效性，通过人的评价，这表明它产生更自然和同步的语音信号相比，竞争的方法。最后，我们创建了一个演示，展示了LipVoicer在产生自然，同步和可理解的语音方面的优势，为其有效性提供了额外的证据。项目页面：https: lipvoicer.github.io摘要：Lip-to-speech involves generating a natural-sounding speech synchronized with a soundless video of a person talking. Despite recent advances, current methods still cannot produce high-quality speech with high levels of intelligibility for challenging and realistic datasets such as LRS3. In this work, we present LipVoicer, a novel method that generates high-quality speech, even for in-the-wild and rich datasets, by incorporating the text modality. Given a silent video, we first predict the spoken text using a pre-trained lip-reading network. We then condition a diffusion model on the video and use the extracted text through a classifier-guidance mechanism where a pre-trained ASR serves as the classifier. LipVoicer outperforms multiple lip-to-speech baselines on LRS2 and LRS3, which are in-the-wild datasets with hundreds of unique speakers in their test set and an unrestricted vocabulary. Moreover, our experiments show that the inclusion of the text modality plays a major role in the intelligibility of the produced speech, readily perceptible while listening, and is empirically reflected in the substantial reduction of the WER metric. We demonstrate the effectiveness of LipVoicer through human evaluation, which shows that it produces more natural and synchronized speech signals compared to competing methods. Finally, we created a demo showcasing LipVoicer's superiority in producing natural, synchronized, and intelligible speech, providing additional evidence of its effectiveness. Project page: https: lipvoicer.github.io
eess.AS音频处理
【1】 On the Robustness of Arabic Speech Dialect Identification标题：阿拉伯语音方言识别的稳健性研究链接：https://arxiv.org/abs/2306.03789
作者：Peter Sullivan,AbdelRahim Elmadany,Muhammad Abdul-Mageed机构：The University of British Columbia, Canada, Mohamed bin Zayed University of Artificial Intelligence, UAE摘要：阿拉伯语方言识别（ADI）工具是训练语音识别模型所需的大规模数据收集管道的重要组成部分。由于这些管道需要将ADI工具应用于潜在的域外数据，因此我们的目标是研究这些工具对这种域转移的脆弱程度。以自监督学习（SSL）模型为起点，我们评估了SSL特征的迁移学习和直接分类。我们在丰富的条件下进行评估，目标是从预训练模型开发ADI系统，并最终评估新收集数据的性能。为了了解哪些因素有助于模型决策，我们对数据的一个子集进行了仔细的人工研究。我们的分析证实，域转移是ADI模型的一个主要挑战。我们还发现，虽然自我训练确实减轻了这种挑战，但它可能不足以满足现实条件。摘要：Arabic dialect identification (ADI) tools are an important part of the large-scale data collection pipelines necessary for training speech recognition models. As these pipelines require application of ADI tools to potentially out-of-domain data, we aim to investigate how vulnerable the tools may be to this domain shift. With self-supervised learning (SSL) models as a starting point, we evaluate transfer learning and direct classification from SSL features. We undertake our evaluation under rich conditions, with a goal to develop ADI systems from pretrained models and ultimately evaluate performance on newly collected data. In order to understand what factors contribute to model decisions, we carry out a careful human study of a subset of our data. Our analysis confirms that domain shift is a major challenge for ADI models. We also find that while self-training does alleviate this challenges, it may be insufficient for realistic conditions.
【2】 Streaming Speech-to-Confusion Network Speech Recognition标题：流式语音到混淆网络语音识别链接：https://arxiv.org/abs/2306.03778
作者：Denis Filimonov,Prabhat Pandey,Ariya Rastrow,Ankur Gandhe,Andreas Stolcke机构：Amazon Alexa备注：Submitted to Interspeech 2023摘要：在交互式自动语音识别（ASR）系统中，低延迟要求限制了在解码期间可以探索的搜索空间的量，特别是在端到端神经ASR中。在本文中，我们提出了一种新的流ASR架构，输出一个混乱的网络，同时保持有限的延迟，需要交互式应用程序。我们表明，我们的模型的1-最佳结果与可比较的RNN-T系统相当，而更丰富的假设集允许第二遍重新评分，以实现LibriSpeech任务上10- 20%的单词错误率。我们还表明，我们的模型在远场语音助理任务上的表现优于强大的RNN-T基线。摘要：In interactive automatic speech recognition (ASR) systems, low-latency requirements limit the amount of search space that can be explored during decoding, particularly in end-to-end neural ASR. In this paper, we present a novel streaming ASR architecture that outputs a confusion network while maintaining limited latency, as needed for interactive applications. We show that 1-best results of our model are on par with a comparable RNN-T system, while the richer hypothesis set allows second-pass rescoring to achieve 10-20 % lower word error rate on the LibriSpeech task. We also show that our model outperforms a strong RNN-T baseline on a far-field voice assistant task.
【3】 Some voices are too common: Building fair speech recognition systems  using the Common Voice dataset标题：有些声音太常见了：使用通用声音数据集构建公平的语音识别系统链接：https://arxiv.org/abs/2306.03773
*与cs.SD语音【9】为同一篇
作者：Lucas Maison,Yannick Estève机构：Laboratoire Informatique d’Avignon, Avignon, France,  Thales SIX, Multimedia Lab, Gennevilliers, France备注：5 pages, 3 figures. Accepted to Interspeech 2023摘要：自动语音识别（ASR）系统变得越来越高效，这要归功于神经网络训练的新进展，如自监督学习。然而，他们被认为是不公平的某些群体，例如，人们说话的口音。在这项工作中，我们使用法国普通语音数据集来量化预训练的wav 2 vec ~2.0模型对几个人口统计群体的偏差。通过在各种固定大小、精心制作的训练集上对预训练模型进行微调，我们证明了说话人多样性的重要性。我们还对通用语音语料库进行了深入分析，并确定了该数据集用户应考虑的重要缺点。摘要：Automatic speech recognition (ASR) systems become increasingly efficient thanks to new advances in neural network training like self-supervised learning. However, they are known to be unfair toward certain groups, for instance, people speaking with an accent. In this work, we use the French Common Voice dataset to quantify the biases of a pre-trained wav2vec~2.0 model toward several demographic groups. By fine-tuning the pre-trained model on a variety of fixed-size, carefully crafted training sets, we demonstrate the importance of speaker diversity. We also run an in-depth analysis of the Common Voice corpus and identify important shortcomings that should be taken into account by users of this dataset.
【4】 Experimenting with Additive Margins for Contrastive Self-Supervised  Speaker Verification标题：基于加性余量的对比自监督说话人确认实验链接：https://arxiv.org/abs/2306.03664
作者：Theo Lepage,Reda Dehak备注：accepted at INTERSPEECH 2023, 20th-24th August 2023, Dublin, Ireland摘要：大多数最先进的自监督说话人确认系统依赖于一个基于对比度的目标函数来学习说话人表示从未标记的语音数据。我们探索不同的方法来提高这些方法的性能：（1）重新审视如何通过对比损失的“对称”公式对正对和负对进行采样;（2）引入类似于在监督设置中广泛采用的AM-Softmax和AAM-Softmax的余量。我们证明了对称对比损失的有效性，它为自监督任务提供了更多的监督。此外，我们表明，添加剂的利润率和添加剂的角度利润率允许通过提高扬声器的可分离性，减少误报和漏报的总数。最后，通过结合这两种技术并训练一个更大的模型，我们在VoxCeleb 1测试集上实现了7.50%EER和0.5804 minDCF，这在说话人验证方面优于其他对比自监督方法。摘要：Most state-of-the-art self-supervised speaker verification systems rely on a contrastive-based objective function to learn speaker representations from unlabeled speech data. We explore different ways to improve the performance of these methods by: (1) revisiting how positive and negative pairs are sampled through a "symmetric" formulation of the contrastive loss; (2) introducing margins similar to AM-Softmax and AAM-Softmax that have been widely adopted in the supervised setting. We demonstrate the effectiveness of the symmetric contrastive loss which provides more supervision for the self-supervised task. Moreover, we show that Additive Margin and Additive Angular Margin allow reducing the overall number of false negatives and false positives by improving speaker separability. Finally, by combining both techniques and training a larger model we achieve 7.50% EER and 0.5804 minDCF on the VoxCeleb1 test set, which outperforms other contrastive self supervised methods on speaker verification.
【5】 Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive  Bias标题：Mega-TTS：具有内在感应偏差的大规模零命中率文本到语音转换链接：https://arxiv.org/abs/2306.03509
*与cs.SD语音【10】为同一篇
作者：Ziyue Jiang,Yi Ren,Zhenhui Ye,Jinglin Liu,Chen Zhang,Qian Yang,Shengpeng Ji,Rongjie Huang,Chunfeng Wang,Xiang Yin,Zejun Ma,Zhou Zhao机构：♠Zhejiang University & ♡ByteDance摘要：将文本到语音缩放到大型和野生数据集已被证明在实现音色和语音风格泛化方面非常有效，特别是在zero-shot TTS中。然而，以往的工作通常使用音频编解码器将语音编码成潜在的，并使用自回归语言模型或扩散模型来生成它，这忽略了语音的内在本质，并可能导致劣质或不可控的结果。我们认为语音可以分解为几个属性（例如，内容、音色、韵律和相位），并且它们中的每一个都应当使用具有适当归纳偏差的模块来建模。从这个角度出发，我们精心设计了一个新颖的大型zero-shot TTS系统，称为Mega-TTS，它使用大规模野生数据进行训练，并以不同的方式对不同的属性进行建模：1）代替使用由音频编解码器编码的潜在特征作为中间特征，我们仍然选择频谱图，因为它很好地分离了相位和其他属性。相位可以由基于GAN的声码器适当地构造，并且不需要由语言模型建模。2)我们使用全局向量对音色进行建模，因为音色是随时间缓慢变化的全局属性。3)我们进一步使用基于VQGAN的声学模型来生成声谱图和潜在代码语言模型来拟合韵律的分布，因为韵律在句子中随着时间的推移而快速变化，并且语言模型可以捕获本地和远程依赖关系。我们将Mega-TTS扩展到具有20 K小时语音的多域数据集，并评估其在看不见的扬声器上的性能。实验结果表明，Mega-TTS超越国家的最先进的TTS系统的zero-shot TTS，语音编辑，跨语言的TTS任务，具有优越的自然性，鲁棒性，和扬声器相似性，由于适当的归纳偏见的每个模块。音频样本可在www.example.com上获得https: mega-tts.github.io demo-page。摘要：Scaling text-to-speech to a large and wild dataset has been proven to be highly effective in achieving timbre and speech style generalization, particularly in zero-shot TTS. However, previous works usually encode speech into latent using audio codec and use autoregressive language models or diffusion models to generate it, which ignores the intrinsic nature of speech and may lead to inferior or uncontrollable results. We argue that speech can be decomposed into several attributes (e.g., content, timbre, prosody, and phase) and each of them should be modeled using a module with appropriate inductive biases. From this perspective, we carefully design a novel and large zero-shot TTS system called Mega-TTS, which is trained with large-scale wild data and models different attributes in different ways: 1) Instead of using latent encoded by audio codec as the intermediate feature, we still choose spectrogram as it separates the phase and other attributes very well. Phase can be appropriately constructed by the GAN-based vocoder and does not need to be modeled by the language model. 2) We model the timbre using global vectors since timbre is a global attribute that changes slowly over time. 3) We further use a VQGAN-based acoustic model to generate the spectrogram and a latent code language model to fit the distribution of prosody, since prosody changes quickly over time in a sentence, and language models can capture both local and long-range dependencies. We scale Mega-TTS to multi-domain datasets with 20K hours of speech and evaluate its performance on unseen speakers. Experimental results demonstrate that Mega-TTS surpasses state-of-the-art TTS systems on zero-shot TTS, speech editing, and cross-lingual TTS tasks, with superior naturalness, robustness, and speaker similarity due to the proper inductive bias of each module. Audio samples are available at https: mega-tts.github.io demo-page.
【6】 A generative framework for conversational laughter: Its 'language model'  and laughter sound synthesis标题：会话笑声的生成框架及其语言模型和笑声合成链接：https://arxiv.org/abs/2306.03465
作者：Hiroki Mori,Shunya Kimura机构：Utsunomiya University, Japan备注：Submitted to INTERSPEECH摘要：由于会话中笑声的语音和声学表现是高度多样的，笑声合成应该能够适应这种多样性，同时保持高度的可控性。本文提出了一个会话中笑声的生成模型，它可以利用情感维度作为会话语境来产生各种各样的笑声。该模型包括两部分：笑声“音素生成器”，其针对给定的说话者ID和情绪状态生成笑声分量的各种但真实的组合;以及笑声“声音合成器”，其接收笑声音素序列并产生反映说话者的个性和情绪状态的声学特征。听力实验的结果表明，调节手机发电机和声音合成器的情感维度导致在合成的笑声中的感知情感的最有效的控制。摘要：As the phonetic and acoustic manifestations of laughter in conversation are highly diverse, laughter synthesis should be capable of accommodating such diversity while maintaining high controllability. This paper proposes a generative model of laughter in conversation that can produce a wide variety of laughter by utilizing the emotion dimension as a conversational context. The model comprises two parts: the laughter "phones generator," which generates various, but realistic, combinations of laughter components for a given speaker ID and emotional state, and the laughter "sound synthesizer," which receives the laughter phone sequence and produces acoustic features that reflect the speaker's individuality and emotional state. The results of a listening experiment indicated that conditioning both the phones generator and the sound synthesizer on emotion dimensions resulted in the most effective control of the perceived emotion in synthesized laughter.
【7】 LipVoicer: Generating Speech from Silent Videos Guided by Lip Reading标题：LipVoier：由Lip Reading引导的无声视频生成语音链接：https://arxiv.org/abs/2306.03258
*与cs.SD语音【11】为同一篇
作者：Yochai Yemini,Aviv Shamsian,Lior Bracha,Sharon Gannot,Ethan Fetaya机构：Bar-Ilan University摘要：唇转语音涉及生成与人说话的无声视频同步的听起来自然的语音。尽管最近的进展，目前的方法仍然不能产生高质量的语音与高水平的可理解性的挑战性和现实的数据集，如LRS3。在这项工作中，我们提出了LipVoicer，一种新的方法，生成高质量的语音，即使在野外和丰富的数据集，通过将文本模态。给定一个无声视频，我们首先使用预先训练的唇读网络预测口语文本。然后，我们在视频上设置扩散模型，并通过分类器指导机制使用提取的文本，其中预先训练的ASR作为分类器。LipVoicer在LRS2和LRS3上的表现优于多个唇语基线，这是一个野外数据集，在其测试集中有数百个独特的说话者和不受限制的词汇。此外，我们的实验表明，包含文本模态在所产生的语音的可懂度中起着重要作用，在收听时容易察觉，并且根据经验反映在WER度量的大幅降低中。我们证明了LipVoicer的有效性，通过人的评价，这表明它产生更自然和同步的语音信号相比，竞争的方法。最后，我们创建了一个演示，展示了LipVoicer在产生自然，同步和可理解的语音方面的优势，为其有效性提供了额外的证据。项目页面：https: lipvoicer.github.io摘要：Lip-to-speech involves generating a natural-sounding speech synchronized with a soundless video of a person talking. Despite recent advances, current methods still cannot produce high-quality speech with high levels of intelligibility for challenging and realistic datasets such as LRS3. In this work, we present LipVoicer, a novel method that generates high-quality speech, even for in-the-wild and rich datasets, by incorporating the text modality. Given a silent video, we first predict the spoken text using a pre-trained lip-reading network. We then condition a diffusion model on the video and use the extracted text through a classifier-guidance mechanism where a pre-trained ASR serves as the classifier. LipVoicer outperforms multiple lip-to-speech baselines on LRS2 and LRS3, which are in-the-wild datasets with hundreds of unique speakers in their test set and an unrestricted vocabulary. Moreover, our experiments show that the inclusion of the text modality plays a major role in the intelligibility of the produced speech, readily perceptible while listening, and is empirically reflected in the substantial reduction of the WER metric. We demonstrate the effectiveness of LipVoicer through human evaluation, which shows that it produces more natural and synchronized speech signals compared to competing methods. Finally, we created a demo showcasing LipVoicer's superiority in producing natural, synchronized, and intelligible speech, providing additional evidence of its effectiveness. Project page: https: lipvoicer.github.io
【8】 Emotion-Conditioned Melody Harmonization with Hierarchical Variational  Autoencoder标题：基于分层变分自动编码器的情感条件旋律和声链接：https://arxiv.org/abs/2306.03718
*与cs.SD语音【1】为同一篇
作者：Shulei Ji,Xinyu Yang机构：School of computer science and technology, Xi’an Jiaotong University, Xi’an, China备注：Accepted by IEEE SMC 2023摘要：现有的旋律和声模型在提高生成和声的质量方面取得了很大的进展，但大多数都忽略了音乐背后的情感。同时，以前的方法产生的谐波的可变性是不够的。为了解决这些问题，我们提出了一种新的基于LSTM的分层变分自动编码器（LHVAE）调查的情绪条件对旋律和谐的影响，同时提高生成的和谐的质量，并捕捉丰富的变化和弦进行。具体而言，LHVAE在不同的级别（块和酒吧级），以模拟全球和本地的音乐属性的潜在变量和情绪条件。此外，我们在每一步引入了一个基于注意力的旋律上下文向量，以更好地学习旋律和和声之间的对应关系。客观评价的实验结果表明，我们提出的模型优于其他基于LSTM的模型。通过主观评价，我们得出结论，仅仅改变和弦几乎没有改变音乐的整体情感。定性分析表明，我们的模型产生变量谐波的能力。摘要：Existing melody harmonization models have made great progress in improving the quality of generated harmonies, but most of them ignored the emotions beneath the music. Meanwhile, the variability of harmonies generated by previous methods is insufficient. To solve these problems, we propose a novel LSTM-based Hierarchical Variational Auto-Encoder (LHVAE) to investigate the influence of emotional conditions on melody harmonization, while improving the quality of generated harmonies and capturing the abundant variability of chord progressions. Specifically, LHVAE incorporates latent variables and emotional conditions at different levels (piece- and bar-level) to model the global and local music properties. Additionally, we introduce an attention-based melody context vector at each step to better learn the correspondence between melodies and harmonies. Experimental results of the objective evaluation show that our proposed model outperforms other LSTM-based models. Through subjective evaluation, we conclude that only altering the chords hardly changes the overall emotion of the music. The qualitative analysis demonstrates the ability of our model to generate variable harmonies.
【9】 Dance Generation by Sound Symbolic Words标题：有声象征性词语的舞蹈生成链接：https://arxiv.org/abs/2306.03646
*与cs.SD语音【2】为同一篇
作者：Miki Okamura,Naruya Kondo,Tatsuki Fushimi Maki Sakamoto,Yoichi Ochiai机构：TATSUKI FUSHIMI, University of Tsukuba, Japan, MAKI SAKAMOTO, The University of Electro-Communications, Japan摘要：本研究提出一种以拟声词为输入的舞蹈动作生成方法，旨在提高舞蹈动作生成的创造性和多样性。与文字和音乐不同，拟声词通过抽象的词语表达来传达节奏和意义，不受表达的限制，也不需要专门的知识。我们采用AI编舞框架，并采用Sakamoto系统，这是一种专注于音素和音节的拟声词特征提取方法。此外，我们提出了一个新的数据集的40拟声舞蹈运动对通过用户调查收集。我们的研究结果表明，所提出的方法，使更直观的舞蹈生成，并可以创建舞蹈动作，使用声音符号的话，从各种语言，包括那些没有象声词。这突出了跨越不同语言和文化的多样化舞蹈创作的潜力，可供更广泛的观众使用。来自我们模型的定性样本可以在以下位置找到：https: sites.google.com view onomatopoeia-dance home 。摘要：This study introduces a novel approach to generate dance motions using onomatopoeia as input, with the aim of enhancing creativity and diversity in dance generation. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions without constraints on expression and without need for specialized knowledge. We adapt the AI Choreographer framework and employ the Sakamoto system, a feature extraction method for onomatopoeia focusing on phonemes and syllables. Additionally, we present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey. Our results demonstrate that the proposed method enables more intuitive dance generation and can create dance motions using sound-symbolic words from a variety of languages, including those without onomatopoeia. This highlights the potential for diverse dance creation across different languages and cultures, accessible to a wider audience. Qualitative samples from our model can be found at: https: sites.google.com view onomatopoeia-dance home .
【10】 Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis标题：Ada-TTA：面向自适应的高质量文本到对话化身合成链接：https://arxiv.org/abs/2306.03504
*与cs.SD语音【3】为同一篇
作者：Zhenhui Ye,Ziyue Jiang,Yi Ren,Jinglin Liu,Chen Zhang,Xiang Yin,Zejun Ma,Zhou Zhao备注：6 pages, 3 figures摘要：我们感兴趣的是一个新的任务，即低资源的文本到说话的化身。给定只有几分钟长的说话人视频与音轨作为训练数据和任意文本作为驱动输入，我们的目标是合成高质量的说话的人像视频对应的输入文本。该任务在数字人行业具有广阔的应用前景，但由于两个挑战，在技术上尚未实现：（1）对于传统的多说话人文本到语音系统，模仿来自域外音频的音色是具有挑战性的。(2)在训练数据有限的情况下，很难呈现高保真和嘴唇同步的说话化身。本文介绍了自适应文本到说话化身（Ada-TTA），它（1）设计了一个通用的zero-shot多说话人TTS模型，能够很好地分离文本内容、音色和韵律;以及（2）包含神经渲染的最新进展以实现逼真的音频驱动的说话面部视频生成。通过这些设计，我们的方法克服了上述两个挑战，并实现了生成身份保留的语音和真实的说话人视频。实验表明，我们的方法可以合成逼真的，身份保护，视听同步说话的化身视频。摘要：We are interested in a novel task, namely low-resource text-to-talking avatar. Given only a few-minute-long talking person video with the audio track as the training data and arbitrary texts as the driving input, we aim to synthesize high-quality talking portrait videos corresponding to the input text. This task has broad application prospects in the digital human industry but has not been technically achieved yet due to two challenges: (1) It is challenging to mimic the timbre from out-of-domain audio for a traditional multi-speaker Text-to-Speech system. (2) It is hard to render high-fidelity and lip-synchronized talking avatars with limited training data. In this paper, we introduce Adaptive Text-to-Talking Avatar (Ada-TTA), which (1) designs a generic zero-shot multi-speaker TTS model that well disentangles the text content, timbre, and prosody; and (2) embraces recent advances in neural rendering to achieve realistic audio-driven talking face video generation. With these designs, our method overcomes the aforementioned two challenges and achieves to generate identity-preserving speech and realistic talking person video. Experiments demonstrate that our method could synthesize realistic, identity-preserving, and audio-visual synchronized talking avatar videos.
【11】 Automatic Assessment of Oral Reading Accuracy for Reading Diagnostics标题：阅读诊断中口语阅读准确度的自动评估链接：https://arxiv.org/abs/2306.03444
*与cs.SD语音【4】为同一篇
作者：Bo Molenaar,Cristian Tejedor-Garcia,Helmer Strik,Catia Cucchiarini机构：Centre for Language and Speech Technology, Radboud University Nijmegen, Netherlands摘要：利用自动语音识别（ASR）技术对阅读流畅性进行自动评估，对于早期发现阅读困难和及时干预具有巨大的潜力。需要精确的评估工具，尤其是英语以外的语言。在这项研究中，我们评估了六个国家的最先进的ASR为基础的系统，自动评估荷兰语口语阅读的准确性使用Kaldi和Whisper。结果表明，我们最成功的系统与人类评价达成了实质性协议（MCC = .63）。同一系统达到了强制解码置信度得分和单词正确性之间的最高相关性（r = .45）。该系统的语言模型（LM）由人工正字法转录和测试数据的阅读提示组成，这表明在LM中包括阅读错误提高了评估性能。我们讨论了发展自动评估系统的影响，并确定了未来研究的可能途径。摘要：Automatic assessment of reading fluency using automatic speech recognition (ASR) holds great potential for early detection of reading difficulties and subsequent timely intervention. Precise assessment tools are required, especially for languages other than English. In this study, we evaluate six state-of-the-art ASR-based systems for automatically assessing Dutch oral reading accuracy using Kaldi and Whisper. Results show our most successful system reached substantial agreement with human evaluations (MCC = .63). The same system reached the highest correlation between forced decoding confidence scores and word correctness (r = .45). This system's language model (LM) consisted of manual orthographic transcriptions and reading prompts of the test data, which shows that including reading errors in the LM improves assessment performance. We discuss the implications for developing automatic assessment systems and identify possible avenues of future research.
【12】 Alzheimer Disease Classification through ASR-based Transcriptions:  Exploring the Impact of Punctuation and Pauses标题：通过基于ASR的转录进行阿尔茨海默病分类：探索标点符号和停顿的影响链接：https://arxiv.org/abs/2306.03443
*与cs.SD语音【5】为同一篇
作者：Lucía Gómez-Zaragozá,Simone Wills,Cristian Tejedor-Garcia,Javier Marín-Morales,Mariano Alcañiz,Helmer Strik机构：Instituto Universitario de Investigacio´n en Tecnolog´ıa Centrada en el Ser Humano, Universitat, Politecnica de Valencia, Spain,  Radboud University, The Netherlands摘要：阿尔茨海默氏病（AD）是世界上主要的神经退行性疾病，其通常导致交流困难。分析语音可以作为诊断工具来识别病情。最近的ADReSS挑战为AD分类提供了数据集，并强调了手动转录的实用性。在这项研究中，我们使用了新的国家的最先进的自动语音识别（ASR）模型耳语获得的转录，其中还包括自动标点符号。分类模型分别在手动和ASR成绩单上实现了0.854和0.833的测试准确度分数，结合了预训练的FastText单词嵌入和递归神经网络。此外，我们探讨了包括暂停信息和标点符号的转录的影响。我们发现，标点符号在某些情况下只产生轻微的改善，而暂停编码辅助AD分类手动和ASR转录在所有的方法研究。摘要：Alzheimer's Disease (AD) is the world's leading neurodegenerative disease, which often results in communication difficulties. Analysing speech can serve as a diagnostic tool for identifying the condition. The recent ADReSS challenge provided a dataset for AD classification and highlighted the utility of manual transcriptions. In this study, we used the new state-of-the-art Automatic Speech Recognition (ASR) model Whisper to obtain the transcriptions, which also include automatic punctuation. The classification models achieved test accuracy scores of 0.854 and 0.833 combining the pretrained FastText word embeddings and recurrent neural networks on manual and ASR transcripts respectively. Additionally, we explored the influence of including pause information and punctuation in the transcriptions. We found that punctuation only yielded minor improvements in some cases, whereas pause encoding aided AD classification for both manual and ASR transcriptions across all approaches investigated.
【13】 Phase perturbation improves channel robustness for speech spoofing  countermeasures标题：相位扰动提高语音欺骗对抗的信道稳健性链接：https://arxiv.org/abs/2306.03389
*与cs.SD语音【6】为同一篇
作者：Yongyi Zang,You Zhang,Zhiyao Duan机构：University of Rochester, Rochester, New York, USA备注：5 pages; Accepted to INTERSPEECH 2023摘要：在本文中，我们的目标是解决语音对抗（CM）系统，这是用来区分合成语音从人类自然语音的信道鲁棒性的问题。在两个假设的基础上，我们提出了一种在时域CM系统的训练过程中扰动相位信息的方法。通信网络通常采用有损压缩编解码器，其仅对幅度信息进行编码，因此严重改变相位信息。此外，现有技术的CM系统依赖于相位信息来识别欺骗语音。因此，我们认为有损压缩编解码器引起的相位域中的信息损失降低了看不见的信道的性能。我们首先建立依赖的时域CM系统的相位信息的扰动相位评估，表现出强烈的退化。然后，我们证明了在训练过程中的扰动阶段导致显着的性能改善，而扰动幅度导致进一步退化。摘要：In this paper, we aim to address the problem of channel robustness in speech countermeasure (CM) systems, which are used to distinguish synthetic speech from human natural speech. On the basis of two hypotheses, we suggest an approach for perturbing phase information during the training of time-domain CM systems. Communication networks often employ lossy compression codec that encodes only magnitude information, therefore heavily altering phase information. Also, state-of-the-art CM systems rely on phase information to identify spoofed speech. Thus, we believe the information loss in the phase domain induced by lossy compression codec degrades the performance of the unseen channel. We first establish the dependence of time-domain CM systems on phase information by perturbing phase in evaluation, showing strong degradation. Then, we demonstrated that perturbing phase during training leads to a significant performance improvement, whereas perturbing magnitude leads to further degradation.
【14】 Reef Elegy: An Auditory Display of Hawaii's 2019 Coral Bleaching Data标题：珊瑚礁悲歌：夏威夷2019年珊瑚白化数据的听觉展示链接：https://arxiv.org/abs/2306.03307
*与cs.SD语音【7】为同一篇
作者：Stefano Kalonaris机构：Music Information Intelligence Team, Center for Advanced Intelligence Project (AIP), RIKEN, Japan备注：To appear in: Proceedings of the 28th International Conference on Auditory Display (ICAD 2023)摘要：本文通过空间音频和参数映射方法描述了夏威夷2019年珊瑚白化数据的听觉显示。选定的数据字段跨越78天被映射到珊瑚礁的自然音景，这是逐步改变其组成元素的相应的珊瑚位置进行漂白的声音代理。对于这些元素中的一些，该过程概述了从密集到稀疏的、减少的音景的轨迹，而对于其他元素，该过程从谐波音调向复杂频谱转变。这个实验是伴随着一个简短的评价研究，它在一个既定的审美视角空间的背景下，并探讨其潜在的公众参与的话语围绕气候变化。摘要：This paper describes an auditory display of Hawaii's 2019 coral bleaching data via means of spatial audio and parameter mapping methods. Selected data fields spanning 78 days are mapped to sound surrogates of coral reefs' natural soundscapes, which are progressively altered in their constituent elements as the corresponding coral locations undergo bleaching. For some of these elements, this process outlines a trajectory from a dense to a sparser, reduced soundscape, while for others it translates moving away from harmonic tones and towards complex spectra. This experiment is accompanied by a short evaluation study to contextualize it in an established aesthetic perspective space and to probe its potential for public engagement in the discourse around climate change.
【15】 DeepVQE: Real Time Deep Voice Quality Enhancement for Joint Acoustic  Echo Cancellation, Noise Suppression and Dereverberation标题：DeepVQE：用于联合声学回波消除、噪声抑制和去混响的实时深音质量增强链接：https://arxiv.org/abs/2306.03177*与cs.SD语音【8】为同一篇
作者：Evgenii Indenbom,Nicolae-Catalin Ristea,Ando Saabas,Tanel Parnamaa,Jegor Guzvin,Ross Cutler机构：Cutler, Microsoft Corp.摘要：声学回声消除（AEC）、噪声抑制（NS）和去混响（DR）是现代全双工通信系统的组成部分。随着对电话会议系统的需求增加，需要解决这些任务以获得有效且高效的在线会议体验。大多数先前的研究分别提出了这些任务的解决方案，将它们与基于数字信号处理（DSP）的组件相结合，导致复杂的管道，这些管道通常在现实世界的应用中部署是不切实际的。本文提出了一种名为DeepVQE的实时交叉注意深度模型，该模型基于残差卷积神经网络（CNN）和递归神经网络（RNN），以同时解决AEC、NS和DR。我们进行了几次消融研究，以分析我们的模型的不同组成部分的整体性能的贡献。DeepVQE在ICASSP 2023声学回声消除挑战赛和ICASSP 2023深度噪声抑制挑战赛测试集的非个性化曲目上实现了最先进的性能，表明单个模型可以以出色的性能处理多个任务。此外，该模型实时运行，并已成功地测试了微软团队平台。摘要：Acoustic echo cancellation (AEC), noise suppression (NS) and dereverberation (DR) are an integral part of modern full-duplex communication systems. As the demand for teleconferencing systems increases, addressing these tasks is required for an effective and efficient online meeting experience. Most prior research proposes solutions for these tasks separately, combining them with digital signal processing (DSP) based components, resulting in complex pipelines that are often impractical to deploy in real-world applications. This paper proposes a real-time cross-attention deep model, named DeepVQE, based on residual convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to simultaneously address AEC, NS, and DR. We conduct several ablation studies to analyze the contributions of different components of our model to the overall performance. DeepVQE achieves state-of-the-art performance on non-personalized tracks from the ICASSP 2023 Acoustic Echo Cancellation Challenge and ICASSP 2023 Deep Noise Suppression Challenge test sets, showing that a single model can handle multiple tasks with excellent performance. Moreover, the model runs in real-time and has been successfully tested for the Microsoft Teams platform.
机器翻译，仅供参考