今日论文合集：cs.SD语音2篇，eess.AS音频处理3篇。

cs.SD语音

【1】 UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice  Conversion

标题：UnifySpeech：一种零精度文语转换的统一框架

链接：https://arxiv.org/abs/2301.03801

作者：Haogeng Liu,Tao Wang,Ruibo Fu,Jiangyan Yi,Zhengqi Wen,Jianhua Tao
机构：Beijing, China, Department of Automation, Tsinghua University, School of Artificial Intelligence, University of Chinese Academy of Sciences, China
摘要：文本语音转换（TTS）和语音转换（VC）是两个不同的任务，都是针对不同的输入模态生成高质量的说话语音。由于两者的相似性，本文提出了UnifySpeech，首次将TTS和VC纳入统一的框架。该模型基于语音可以被解耦成三个独立分量的假设：内容信息、说话人信息、韵律信息。TTS和VC都可以看作是从输入中挖掘这三部分信息，完成语音的重构。TTS的语音内容信息来源于文本，而VC的语音内容信息来源于源语音，因此两个任务中除了语音内容提取模块外，其余所有单元都是共享的。我们利用向量量化和领域限制技术来弥补TTS和VC在内容领域上的差距。客观和主观评价表明，通过将两者结合起来，TTS具有较好的说话人建模能力，VC具有较好的语音内容解耦能力。
摘要：Text-to-speech (TTS) and voice conversion (VC) are two different tasks both aiming at generating high quality speaking voice according to different input modality. Due to their similarity, this paper proposes UnifySpeech, which brings TTS and VC into a unified framework for the first time. The model is based on the assumption that speech can be decoupled into three independent components: content information, speaker information, prosody information. Both TTS and VC can be regarded as mining these three parts of information from the input and completing the reconstruction of speech. For TTS, the speech content information is derived from the text, while in VC it's derived from the source speech, so all the remaining units are shared except for the speech content extraction module in the two tasks. We applied vector quantization and domain constrain to bridge the gap between the content domains of TTS and VC. Objective and subjective evaluation shows that by combining the two task, TTS obtains better speaker modeling ability while VC gets hold of impressive speech content decoupling capability.


【2】 Generative Emotional AI for Speech Emotion Recognition: The Case for  Synthetic Emotional Speech Augmentation

标题：用于语音情感识别的生成性情感人工智能：合成情感语音增强

链接：https://arxiv.org/abs/2301.03751

作者：Abdullah Shahid,Siddique Latif,Junaid Qadir
机构：Information Technology University (ITU),Punjab, Pakistan, University of Southern Queensland, Australia, Qatar University, Doha, Qatar, A R T I C L E I N F O
备注：Under review
摘要：尽管在深度学习方面取得了进展，但由于缺乏语音情感数据集，当前最先进的语音情感识别（SER）系统仍然具有较差的性能。本文提出用基于扩展的Tacotron架构的端到端文本到语音（TTS）系统生成的合成情感语音来增强SER系统。该TTS系统包括用于说话人和情感嵌入的编码器、用于创建Mel谱图的序列到序列文本生成器以及用于从Mel谱图生成音频的WaveRNN。大量实验表明，情感语音的质量可以显著提高多个数据集上的SER性能，与基线相比，平均意见得分（MOS）更高。所生成的样本在增强SER性能方面也是有效的。
摘要：Despite advances in deep learning, current state-of-the-art speech emotion recognition (SER) systems still have poor performance due to a lack of speech emotion datasets. This paper proposes augmenting SER systems with synthetic emotional speech generated by an end-to-end text-to-speech (TTS) system based on an extended Tacotron architecture. The proposed TTS system includes encoders for speaker and emotion embeddings, a sequence-to-sequence text generator for creating Mel-spectrograms, and a WaveRNN to generate audio from the Mel-spectrograms. Extensive experiments show that the quality of the generated emotional speech can significantly improve SER performance on multiple datasets, as demonstrated by a higher mean opinion score (MOS) compared to the baseline. The generated samples were also effective at augmenting SER performance.


eess.AS音频处理

【1】 BASPRO: a balanced script producer for speech corpus collection based on  the genetic algorithm

标题：BASPRO：一种基于遗传算法的语音语料库均衡脚本生成器

链接：https://arxiv.org/abs/2301.04120

作者：Yu-Wen Chen,Hsin-Min Wang,Yu Tsao
机构：Taiwan and Department of Computer Science, Columbia University, Hsin-Min Wang is with the Institute of Information Science
备注：accepted by APSIPA Transactions on Signal and Information Processing
摘要：用于训练和评估的语音语料库对语音处理模型的性能有很大影响。在本研究中，我们提出了一个平衡脚本生成器（BASPRO）系统，它可以自动构建一个语音平衡且丰富的汉语句子集，用于收集普通话语音数据。首先，我们使用预先训练好的自然语言处理系统从一个大型中文新闻文本语料库中抽取十字候选句。然后，我们应用一种基于遗传算法的方法从候选句子中选出20个语音平衡的句子集，每个句子集包含20个句子。利用BASPRO得到了一个包含400个10字语句的录音脚本TMNews。TMNews覆盖了现实世界中使用的84%的音节。此外，音节分布与真实世界音节分布具有0.96余弦相似度。我们使用两个文本到语音系统将脚本转换为语音语料库。利用所设计的语音语料库，我们测试了语音增强（SE）和自动语音识别（ASR）的性能，这两项任务分别是最重要的基于回归和分类的语音处理任务之一。实验结果表明，在设计的语音库上训练的SE和ASR模型优于在随机合成的语音库上训练的模型。
摘要：The performance of speech-processing models is heavily influenced by the speech corpus that is used for training and evaluation. In this study, we propose BAlanced Script PROducer (BASPRO) system, which can automatically construct a phonetically balanced and rich set of Chinese sentences for collecting Mandarin Chinese speech data. First, we used pretrained natural language processing systems to extract ten-character candidate sentences from a large corpus of Chinese news texts. Then, we applied a genetic algorithm-based method to select 20 phonetically balanced sentence sets, each containing 20 sentences, from the candidate sentences. Using BASPRO, we obtained a recording script called TMNews, which contains 400 ten-character sentences. TMNews covers 84% of the syllables used in the real world. Moreover, the syllable distribution has 0.96 cosine similarity to the real-world syllable distribution. We converted the script into a speech corpus using two text-to-speech systems. Using the designed speech corpus, we tested the performances of speech enhancement (SE) and automatic speech recognition (ASR), which are one of the most important regression- and classification-based speech processing tasks, respectively. The experimental results show that the SE and ASR models trained on the designed speech corpus outperform their counterparts trained on a randomly composed speech corpus.


【2】 UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice  Conversion

标题：UnifySpeech：一种零精度文语转换的统一框架

链接：https://arxiv.org/abs/2301.03801

*与cs.SD语音【1】为同一篇

作者：Haogeng Liu,Tao Wang,Ruibo Fu,Jiangyan Yi,Zhengqi Wen,Jianhua Tao
机构：Beijing, China, Department of Automation, Tsinghua University, School of Artificial Intelligence, University of Chinese Academy of Sciences, China
摘要：文本语音转换（TTS）和语音转换（VC）是两个不同的任务，都是针对不同的输入模态生成高质量的说话语音。由于两者的相似性，本文提出了UnifySpeech，首次将TTS和VC纳入统一的框架。该模型基于语音可以被解耦成三个独立分量的假设：内容信息、说话人信息、韵律信息。TTS和VC都可以看作是从输入中挖掘这三部分信息，完成语音的重构。TTS的语音内容信息来源于文本，而VC的语音内容信息来源于源语音，因此两个任务中除了语音内容提取模块外，其余所有单元都是共享的。我们利用向量量化和领域限制技术来弥补TTS和VC在内容领域上的差距。客观和主观评价表明，通过将两者结合起来，TTS具有较好的说话人建模能力，VC具有较好的语音内容解耦能力。
摘要：Text-to-speech (TTS) and voice conversion (VC) are two different tasks both aiming at generating high quality speaking voice according to different input modality. Due to their similarity, this paper proposes UnifySpeech, which brings TTS and VC into a unified framework for the first time. The model is based on the assumption that speech can be decoupled into three independent components: content information, speaker information, prosody information. Both TTS and VC can be regarded as mining these three parts of information from the input and completing the reconstruction of speech. For TTS, the speech content information is derived from the text, while in VC it's derived from the source speech, so all the remaining units are shared except for the speech content extraction module in the two tasks. We applied vector quantization and domain constrain to bridge the gap between the content domains of TTS and VC. Objective and subjective evaluation shows that by combining the two task, TTS obtains better speaker modeling ability while VC gets hold of impressive speech content decoupling capability.


【3】 Generative Emotional AI for Speech Emotion Recognition: The Case for  Synthetic Emotional Speech Augmentation

标题：用于语音情感识别的生成性情感人工智能：合成情感语音增强

链接：https://arxiv.org/abs/2301.03751

*与cs.SD语音【2】为同一篇

作者：Abdullah Shahid,Siddique Latif,Junaid Qadir
机构：Information Technology University (ITU),Punjab, Pakistan, University of Southern Queensland, Australia, Qatar University, Doha, Qatar, A R T I C L E I N F O
备注：Under review
摘要：尽管在深度学习方面取得了进展，但由于缺乏语音情感数据集，当前最先进的语音情感识别（SER）系统仍然具有较差的性能。本文提出用基于扩展的Tacotron架构的端到端文本到语音（TTS）系统生成的合成情感语音来增强SER系统。该TTS系统包括用于说话人和情感嵌入的编码器、用于创建Mel谱图的序列到序列文本生成器以及用于从Mel谱图生成音频的WaveRNN。大量实验表明，情感语音的质量可以显著提高多个数据集上的SER性能，与基线相比，平均意见得分（MOS）更高。所生成的样本在增强SER性能方面也是有效的。
摘要：Despite advances in deep learning, current state-of-the-art speech emotion recognition (SER) systems still have poor performance due to a lack of speech emotion datasets. This paper proposes augmenting SER systems with synthetic emotional speech generated by an end-to-end text-to-speech (TTS) system based on an extended Tacotron architecture. The proposed TTS system includes encoders for speaker and emotion embeddings, a sequence-to-sequence text generator for creating Mel-spectrograms, and a WaveRNN to generate audio from the Mel-spectrograms. Extensive experiments show that the quality of the generated emotional speech can significantly improve SER performance on multiple datasets, as demonstrated by a higher mean opinion score (MOS) compared to the baseline. The generated samples were also effective at augmenting SER performance.


机器翻译，仅供参考