今天跟大家分享一篇语音相关的论文合集：cs.SD语音4篇，eess.AS音频处理4篇。
【1】 Deploying Enhanced Speech Feature Decreased Audio Complaints at SVT Play VOD Service
标题：部署增强的语音功能减少了SVT Play VOD服务的音频投诉
链接：https://arxiv.org/abs/2208.08960
作者：Annika Bidner,Julia Lindberg,Olof Lindman,Kinga Skorupska备注：9 pages, study based on a practical implementation at SVT摘要：At Public Service Broadcaster SVT in Sweden, background music and sounds in programs have for many years been one of the most common complaints from the viewers. The most sensitive group are people with hearing disabilities, but many others also find background sounds annoying. To address this problem SVT has added Enhanced Speech, a feature with lower background noise, to a number of TV programs in VOD service SVT Play. As a result, when the number of programs with the Enhanced Speech feature increased, the level of audio complaints to customer service decreased. The Enhanced Speech feature got the rating 8.3/10 in a survey with 86 participants. The rating for possible future usage was 9.0/10. In this article we describe this feature's design and development process, as well as its technical specification, limitations and future development opportunities.
【2】 Musika! Fast Infinite Waveform Music Generation
标题：快速无限波形音乐生成
链接：https://arxiv.org/abs/2208.08706
作者：Marco Pasini,Jan Schlüter机构：Institute of Computational Perception, Johannes Kepler University Linz, Austria备注：Accepted at ISMIR 2022摘要：Fast and user-controllable music generation could enable novel ways of composing or performing music. However, state-of-the-art music generation systems require large amounts of data and computational resources for training, and are slow at inference. This makes them impractical for real-time interactive use. In this work, we introduce Musika, a music generation system that can be trained on hundreds of hours of music using a single consumer GPU, and that allows for much faster than real-time generation of music of arbitrary length on a consumer CPU. We achieve this by first learning a compact invertible representation of spectrogram magnitudes and phases with adversarial autoencoders, then training a Generative Adversarial Network (GAN) on this representation for a particular music domain. A latent coordinate system enables generating arbitrarily long sequences of excerpts in parallel, while a global context vector allows the music to remain stylistically coherent through time. We perform quantitative evaluations to assess the quality of the generated samples and showcase options for user control in piano and techno music generation. We release the source code and pretrained autoencoder weights at github.com/marcoppasini/musika, such that a GAN can be trained on a new music domain with a single GPU in a matter of hours.
【3】 Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition
标题：自动语音识别中端到端神经模型的鲁棒性分析
链接：https://arxiv.org/abs/2208.08509
作者：Goutham Rajendran,Wei Zou摘要：We investigate robustness properties of pre-trained neural models for automatic speech recognition. Real life data in machine learning is usually very noisy and almost never clean, which can be attributed to various factors depending on the domain, e.g. outliers, random noise and adversarial noise. Therefore, the models we develop for various tasks should be robust to such kinds of noisy data, which led to the thriving field of robust machine learning. We consider this important issue in the setting of automatic speech recognition. With the increasing popularity of pre-trained models, it's an important question to analyze and understand the robustness of such models to noise. In this work, we perform a robustness analysis of the pre-trained neural models wav2vec2, HuBERT and DistilHuBERT on the LibriSpeech and TIMIT datasets. We use different kinds of noising mechanisms and measure the model performances as quantified by the inference time and the standard Word Error Rate metric. We also do an in-depth layer-wise analysis of the wav2vec2 model when injecting noise in between layers, enabling us to predict at a high level what each layer learns. Finally for this model, we visualize the propagation of errors across the layers and compare how it behaves on clean versus noisy data. Our experiments conform the predictions of Pasad et al. [2021] and also raise interesting directions for future work.
【4】 Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversion
标题：基于对抗互信息学习的单次语音转换中语音表示解纠缠
链接：https://arxiv.org/abs/2208.08757
作者：SiCheng Yang,Methawee Tantrawenith,Haolin Zhuang,Zhiyong Wu,Aolan Sun,Jianzong Wang,ning cheng,Huaizhen Tang,Xintao Zhao,Jie Wang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, Ping An Technology (Shenzhen) Co., Ltd., China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：5 pages,5 figures,INTERSPEECH 2022摘要：One-shot voice conversion (VC) with only a single target speaker's speech for reference has become a hot research topic. Existing works generally disentangle timbre, while information about pitch, rhythm and content is still mixed together. To perform one-shot VC effectively with further disentangling these speech components, we employ random resampling for pitch and content encoder and use the variational contrastive log-ratio upper bound of mutual information and gradient reversal layer based adversarial mutual information learning to ensure the different parts of the latent space containing only the desired disentangled representation during training. Experiments on the VCTK dataset show the model achieves state-of-the-art performance for one-shot VC in terms of naturalness and intellgibility. In addition, we can transfer characteristics of one-shot VC on timbre, pitch and rhythm separately by speech representation disentanglement. Our code, pre-trained models and demo are available at https://im1eon.github.io/IS2022-SRDVC/.
【1】 Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversion
标题：基于对抗互信息学习的单次语音转换中语音表示解纠缠
链接：https://arxiv.org/abs/2208.08757
* 与cs.SD语音【4】为同一篇
作者：SiCheng Yang,Methawee Tantrawenith,Haolin Zhuang,Zhiyong Wu,Aolan Sun,Jianzong Wang,ning cheng,Huaizhen Tang,Xintao Zhao,Jie Wang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, Ping An Technology (Shenzhen) Co., Ltd., China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：5 pages,5 figures,INTERSPEECH 2022摘要：One-shot voice conversion (VC) with only a single target speaker's speech for reference has become a hot research topic. Existing works generally disentangle timbre, while information about pitch, rhythm and content is still mixed together. To perform one-shot VC effectively with further disentangling these speech components, we employ random resampling for pitch and content encoder and use the variational contrastive log-ratio upper bound of mutual information and gradient reversal layer based adversarial mutual information learning to ensure the different parts of the latent space containing only the desired disentangled representation during training. Experiments on the VCTK dataset show the model achieves state-of-the-art performance for one-shot VC in terms of naturalness and intellgibility. In addition, we can transfer characteristics of one-shot VC on timbre, pitch and rhythm separately by speech representation disentanglement. Our code, pre-trained models and demo are available at https://im1eon.github.io/IS2022-SRDVC/.
【2】 Deploying Enhanced Speech Feature Decreased Audio Complaints at SVT Play VOD Service
标题：部署增强的语音功能减少了SVT Play VOD服务的音频投诉
链接：https://arxiv.org/abs/2208.08960
* 与cs.SD语音【1】为同一篇
作者：Annika Bidner,Julia Lindberg,Olof Lindman,Kinga Skorupska备注：9 pages, study based on a practical implementation at SVT摘要：At Public Service Broadcaster SVT in Sweden, background music and sounds in programs have for many years been one of the most common complaints from the viewers. The most sensitive group are people with hearing disabilities, but many others also find background sounds annoying. To address this problem SVT has added Enhanced Speech, a feature with lower background noise, to a number of TV programs in VOD service SVT Play. As a result, when the number of programs with the Enhanced Speech feature increased, the level of audio complaints to customer service decreased. The Enhanced Speech feature got the rating 8.3/10 in a survey with 86 participants. The rating for possible future usage was 9.0/10. In this article we describe this feature's design and development process, as well as its technical specification, limitations and future development opportunities.
【3】 Musika! Fast Infinite Waveform Music Generation
标题：快速无限波形音乐生成
链接：https://arxiv.org/abs/2208.08706
* 与cs.SD语音【2】为同一篇
作者：Marco Pasini,Jan Schlüter机构：Institute of Computational Perception, Johannes Kepler University Linz, Austria备注：Accepted at ISMIR 2022摘要：Fast and user-controllable music generation could enable novel ways of composing or performing music. However, state-of-the-art music generation systems require large amounts of data and computational resources for training, and are slow at inference. This makes them impractical for real-time interactive use. In this work, we introduce Musika, a music generation system that can be trained on hundreds of hours of music using a single consumer GPU, and that allows for much faster than real-time generation of music of arbitrary length on a consumer CPU. We achieve this by first learning a compact invertible representation of spectrogram magnitudes and phases with adversarial autoencoders, then training a Generative Adversarial Network (GAN) on this representation for a particular music domain. A latent coordinate system enables generating arbitrarily long sequences of excerpts in parallel, while a global context vector allows the music to remain stylistically coherent through time. We perform quantitative evaluations to assess the quality of the generated samples and showcase options for user control in piano and techno music generation. We release the source code and pretrained autoencoder weights at github.com/marcoppasini/musika, such that a GAN can be trained on a new music domain with a single GPU in a matter of hours.
【4】 Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition
标题：自动语音识别中端到端神经模型的鲁棒性分析
链接：https://arxiv.org/abs/2208.08509
* 与cs.SD语音【3】为同一篇
作者：Goutham Rajendran,Wei Zou摘要：We investigate robustness properties of pre-trained neural models for automatic speech recognition. Real life data in machine learning is usually very noisy and almost never clean, which can be attributed to various factors depending on the domain, e.g. outliers, random noise and adversarial noise. Therefore, the models we develop for various tasks should be robust to such kinds of noisy data, which led to the thriving field of robust machine learning. We consider this important issue in the setting of automatic speech recognition. With the increasing popularity of pre-trained models, it's an important question to analyze and understand the robustness of such models to noise. In this work, we perform a robustness analysis of the pre-trained neural models wav2vec2, HuBERT and DistilHuBERT on the LibriSpeech and TIMIT datasets. We use different kinds of noising mechanisms and measure the model performances as quantified by the inference time and the standard Word Error Rate metric. We also do an in-depth layer-wise analysis of the wav2vec2 model when injecting noise in between layers, enabling us to predict at a high level what each layer learns. Finally for this model, we visualize the propagation of errors across the layers and compare how it behaves on clean versus noisy data. Our experiments conform the predictions of Pasad et al. [2021] and also raise interesting directions for future work.
每日语音速递