每日语音速递cs.SD语音
【1】 Retrieval Augmented Generation of Symbolic Music with LLMs标题：基于LLMS的符号音乐检索增强生成链接：https://arxiv.org/abs/2311.10384
作者：Nicolas Jonason,Luca Casini,Carl Thomé,Bob L. T. Sturm备注：LBD @ ISMIR 2023摘要：我们探索使用大型语言模型（LLM）的音乐生成使用检索系统来选择相关的例子。我们发现有希望的初步结果，在与用户的对话中的音乐生成，特别是考虑到这种系统可以实现的容易性。代码可在线获取。摘要：We explore the use of large language models (LLMs) for music generation using a retrieval system to select relevant examples. We find promising initial results for music generation in a dialogue with the user, especially considering the ease with which such a system can be implemented. The code is available online.
【2】 AQUATK: An Audio Quality Assessment Toolkit标题：AQUATK：音频质量评估工具包链接：https://arxiv.org/abs/2311.10113
作者：Ashvala Vinay,Alexander Lerch摘要：神经音频合成（NAS）的最新进展已经超过了标准化评估方法和工具的发展。为了弥补这一差距，我们引入了AquaTk，这是一个开源Python库，专门用于简化和标准化NAS系统的评估。AquaTk提供了一系列音频质量指标，包括基本PEAQ算法的独特Python实现，并在多种模式下运行，以满足各种用户需求。摘要：Recent advancements in Neural Audio Synthesis (NAS) have outpaced the development of standardized evaluation methodologies and tools. To bridge this gap, we introduce AquaTk, an open-source Python library specifically designed to simplify and standardize the evaluation of NAS systems. AquaTk offers a range of audio quality metrics, including a unique Python implementation of the basic PEAQ algorithm, and operates in multiple modes to accommodate various user needs.
【1】 GhostVec: A New Threat to Speaker Privacy of End-to-End Speech Recognition System标题：Ghost Vec：端到端语音识别系统对说话人隐私的新威胁链接：https://arxiv.org/abs/2311.10689
作者：Xiaojiao Chen,Sheng Li,Jiyi Li,Hao Huang,Yang Cao,Liang He备注：accepted in ACM Multimedia Asia 2023摘要：说话人自适应系统面临着隐私问题，因为这种系统是在私人数据集上训练的，并且经常过拟合。本文证明了攻击者可以通过查询说话人自适应语音识别（ASR）系统来提取说话人信息。我们专注于基于transformer的ASR的说话人信息，并提出GhostVec，一个简单而有效的攻击方法来提取说话人信息的编码器-解码器为基础的ASR系统没有任何外部的说话人验证系统或自然人的声音作为参考。为了使我们的结果定量化，我们使用奇异值分解（SVD）对GhostVec进行预处理，并将其合成为波形。实验结果表明，GhostVec语音合成的EER达到10.83%，与目标说话人的DCF达到0.47 min.我们希望这项研究的初步发现能促进未来语音识别在隐私保护方面的研究。摘要：Speaker adaptation systems face privacy concerns, for such systems are trained on private datasets and often overfitting. This paper demonstrates that an attacker can extract speaker information by querying speaker-adapted speech recognition (ASR) systems. We focus on the speaker information of a transformer-based ASR and propose GhostVec, a simple and efficient attack method to extract the speaker information from an encoder-decoder-based ASR system without any external speaker verification system or natural human voice as a reference. To make our results quantitative, we pre-process GhostVec using singular value decomposition (SVD) and synthesize it into waveform. Experiment results show that the synthesized audio of GhostVec reaches 10.83\% EER and 0.47 minDCF with target speakers, which suggests the effectiveness of the proposed method. We hope the preliminary discovery in this study to catalyze future speech recognition research on privacy-preserving topics.
【2】 Reprogramming Self-supervised Learning-based Speech Representations for Speaker Anonymization标题：用于说话人匿名化的基于自监督学习的重编程语音表示链接：https://arxiv.org/abs/2311.10664
作者：Xiaojiao Chen,Sheng Li,Jiyi Li,Hao Huang,Yang Cao,Liang He备注：accepted in ACM Multimedia Asia2023摘要：目前的说话人匿名方法，特别是自监督学习（SSL）模型，需要大量的计算资源时隐藏说话人身份。本文提出了一种基于端到端模型重编程技术的高效、参数节省的说话人匿名方法。为了提高匿名化的性能，我们首先从大型SSL模型中提取说话人表示作为说话人标识。为了隐藏说话人的身份，我们重新编程的扬声器表示适应扬声器的伪域。在VoicePrivacy Challenge（VPC）2022数据集上进行了大量实验，以证明我们提出的参数高效学习匿名方法的有效性。此外，在实现与VPC 2022强基线1.b相当的性能的同时，我们的方法在匿名化过程中消耗的计算资源更少。摘要：Current speaker anonymization methods, especially with self-supervised learning (SSL) models, require massive computational resources when hiding speaker identity. This paper proposes an effective and parameter-efficient speaker anonymization method based on recent End-to-End model reprogramming technology. To improve the anonymization performance, we first extract speaker representation from large SSL models as the speaker identifies. To hide the speaker's identity, we reprogram the speaker representation by adapting the speaker to a pseudo domain. Extensive experiments are carried out on the VoicePrivacy Challenge (VPC) 2022 datasets to demonstrate the effectiveness of our proposed parameter-efficient learning anonymization methods. Additionally, while achieving comparable performance with the VPC 2022 strong baseline 1.b, our approach consumes less computational resources during anonymization.
【3】 LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement标题：LE-SSL-MOS：基于听者增强的自监督学习MOS预测链接：https://arxiv.org/abs/2311.10656
作者：Zili Qi,Xinhui Hu,Wangjin Zhou,Sheng Li,Hao Wu,Jian Lu,Xinkang Xu备注：accepted in IEEE-ASRU2023摘要：近年来，研究人员对自动预测语音合成系统的主观评价表现出越来越大的兴趣。这种预测是一项具有挑战性的任务，特别是在域外测试集上。在本文中，我们提出了一种新的融合模型MOS预测相结合的监督和无监督的方法。在监督方面，我们开发了一个基于SSL的预测器LE-SSL-MOS。LE-SSL-MOS利用预训练的自监督学习模型，并通过利用收听者增强分支中每个话语的意见分数来进一步提高预测准确性。在无监督方面，包含两个步骤：我们使用高度可理解的领域数据微调单元语言模型（ULM），以提高无监督度量的相关性- SpeechLMScore。另一个是我们利用ASR信心作为一个新的度量与集成学习的帮助。据我们所知，这是第一个融合监督和无监督方法进行MOS预测的架构。通过这些方法，我们在VoiceMOS Challenge 2023上的实验结果表明，LE-SSL-MOS的性能优于基线。我们的融合系统实现了13%的绝对改善，在噪声和增强语音跟踪的LE-SSL-MOS。我们的系统分别在法语语音合成赛道和挑战赛的嘈杂和增强语音赛道中排名第一和第二。摘要：Recently, researchers have shown an increasing interest in automatically predicting the subjective evaluation for speech synthesis systems. This prediction is a challenging task, especially on the out-of-domain test set. In this paper, we proposed a novel fusion model for MOS prediction that combines supervised and unsupervised approaches. In the supervised aspect, we developed an SSL-based predictor called LE-SSL-MOS. The LE-SSL-MOS utilizes pre-trained self-supervised learning models and further improves prediction accuracy by utilizing the opinion scores of each utterance in the listener enhancement branch. In the unsupervised aspect, two steps are contained: we fine-tuned the unit language model (ULM) using highly intelligible domain data to improve the correlation of an unsupervised metric - SpeechLMScore. Another is that we utilized ASR confidence as a new metric with the help of ensemble learning. To our knowledge, this is the first architecture that fuses supervised and unsupervised methods for MOS prediction. With these approaches, our experimental results on the VoiceMOS Challenge 2023 show that LE-SSL-MOS performs better than the baseline. Our fusion system achieved an absolute improvement of 13% over LE-SSL-MOS on the noisy and enhanced speech track. Our system ranked 1st and 2nd, respectively, in the French speech synthesis track and the challenge's noisy and enhanced speech track.
【4】 Improving fairness for spoken language understanding in atypical speech with Text-to-Speech标题：利用文语转换提高非典型语音口语理解的公平性链接：https://arxiv.org/abs/2311.10149
作者：Helin Wang,Venkatesh Ravichandran,Milind Rao,Becky Lammers,Myra Sydnor,Nicholas Maragakis,Ankur A. Butala,Jayne Zhang,Lora Clawson,Victoria Chovaz,Laureano Moro-Velazquez备注：Accepted at SyntheticData4ML 2023 Oral摘要：口语理解（SLU）系统在处理非典型语音时通常表现出次优性能，这通常是由神经系统疾病和运动障碍引起的。基于文本到语音（TTS）合成的增强以实现更公平的SLU的最新进展一直难以准确捕获非典型说话者的独特声音特征，这主要是由于数据不足。为了解决这个问题，我们提出了一种新的非典型扬声器的数据增强方法，通过微调的TTS模型，称为Aty-TTS。Aty-TTS通过从语音转换模型转移知识来对说话人和非典型特征进行建模。然后，我们使用增强的数据来训练适应于非典型语音的SLU模型。为了训练这些数据增强模型并评估由此产生的SLU系统，我们收集了一个新的包含意图注释的非典型语音数据集。客观和主观评估都验证了Aty-TTS能够生成高质量的非典型语音。此外，它作为一个有效的数据增强策略，有助于更公平的SLU系统，可以更好地适应个人与非典型的语音模式。摘要：Spoken language understanding (SLU) systems often exhibit suboptimal performance in processing atypical speech, typically caused by neurological conditions and motor impairments. Recent advancements in Text-to-Speech (TTS) synthesis-based augmentation for more fair SLU have struggled to accurately capture the unique vocal characteristics of atypical speakers, largely due to insufficient data. To address this issue, we present a novel data augmentation method for atypical speakers by finetuning a TTS model, called Aty-TTS. Aty-TTS models speaker and atypical characteristics via knowledge transferring from a voice conversion model. Then, we use the augmented data to train SLU models adapted to atypical speech. To train these data augmentation models and evaluate the resulting SLU systems, we have collected a new atypical speech dataset containing intent annotation. Both objective and subjective assessments validate that Aty-TTS is capable of generating high-quality atypical speech. Furthermore, it serves as an effective data augmentation strategy, contributing to more fair SLU systems that can better accommodate individuals with atypical speech patterns.
【5】 Retrieval Augmented Generation of Symbolic Music with LLMs标题：基于LLM的符号音乐检索增强生成链接：https://arxiv.org/abs/2311.10384
作者：Nicolas Jonason,Luca Casini,Carl Thomé,Bob L. T. Sturm备注：LBD @ ISMIR 2023摘要：我们探索使用大型语言模型（LLM）的音乐生成使用检索系统来选择相关的例子。我们发现有希望的初步结果，在与用户的对话中的音乐生成，特别是考虑到这种系统可以实现的容易性。代码可在线获取。摘要：We explore the use of large language models (LLMs) for music generation using a retrieval system to select relevant examples. We find promising initial results for music generation in a dialogue with the user, especially considering the ease with which such a system can be implemented. The code is available online.
【6】 AQUATK: An Audio Quality Assessment Toolkit标题：AQUATK：音频质量评估工具包链接：https://arxiv.org/abs/2311.10113
作者：Ashvala Vinay,Alexander Lerch摘要：神经音频合成（NAS）的最新进展已经超过了标准化评估方法和工具的发展。为了弥补这一差距，我们引入了AquaTk，这是一个开源Python库，专门用于简化和标准化NAS系统的评估。AquaTk提供了一系列音频质量指标，包括基本PEAQ算法的独特Python实现，并在多种模式下运行，以满足各种用户需求。摘要：Recent advancements in Neural Audio Synthesis (NAS) have outpaced the development of standardized evaluation methodologies and tools. To bridge this gap, we introduce AquaTk, an open-source Python library specifically designed to simplify and standardize the evaluation of NAS systems. AquaTk offers a range of audio quality metrics, including a unique Python implementation of the basic PEAQ algorithm, and operates in multiple modes to accommodate various user needs.
机器翻译，仅供参
