语音/音频处理学术速递[1.20]

今日论文合集：cs.SD语音5篇，eess.AS音频处理5篇。

cs.SD语音

【1】 Music Playlist Title Generation Using Artist Information

标题：使用艺术家信息生成音乐播放列表标题

链接：https://arxiv.org/abs/2301.08145

作者：Haven Kim,SeungHeon Doh,Junwon Lee,Juhan Nam

机构：Graduate School of Culture Technology, KAIST, Department of Electrical Engineering, KAIST

备注：AAAI-23 Workshop on Creative AI Across Modalities

摘要：在给定一组曲目的情况下，自动生成音乐播放列表标题或为其添加字幕在音乐流服务中具有显著的兴趣，因为定制的播放列表广泛地用于个性化音乐推荐，并且良好构成的文本标题吸引用户并帮助他们发现音乐。我们提出了一个编码器-解码器模型，它可以从一系列音乐曲目中生成播放列表标题。虽然先前的工作采用曲目ID作为用于播放列表标题生成的标记化输入，但是我们使用对应于曲目的艺术家ID来缓解来自包括在播放列表数据集中的曲目的长尾分布的问题。此外，我们还引入了一种按时间顺序的数据分割方法来处理真实场景中新发布的曲目。比较曲目ID和艺术家ID作为输入序列，我们表明基于艺术家的方法在单词重叠、语义相关性和多样性方面显著提高了性能。

摘要：Automatically generating or captioning music playlist titles given a set of tracks is of significant interest in music streaming services as customized playlists are widely used in personalized music recommendation, and well-composed text titles attract users and help their music discovery. We present an encoder-decoder model that generates a playlist title from a sequence of music tracks. While previous work takes track IDs as tokenized input for playlist title generation, we use artist IDs corresponding to the tracks to mitigate the issue from the long-tail distribution of tracks included in the playlist dataset. Also, we introduce a chronological data split method to deal with newly-released tracks in real-world scenarios. Comparing the track IDs and artist IDs as input sequences, we show that the artist-based approach significantly enhances the performance in terms of word overlap, semantic relevance, and diversity.

【2】 SpotHitPy: A Study For ML-Based Song Hit Prediction Using Spotify

标题：SpotHitPy：基于ML的Spotify歌曲热播预测研究

链接：https://arxiv.org/abs/2301.07978

作者：Ioannis Dimolitsas,Spyridon Kantarelis,Afroditi Fouka

机构：School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece

摘要：在本研究中，我们探讨了热门歌曲预测问题，旨在预测哪些歌曲将成为公告牌热门歌曲。我们收集了近18500首热门和非热门歌曲的数据集，并使用Spotify Web API提取了它们的音频特征。我们在我们的数据集上测试了四个机器学习模型。我们能够以大约86%的准确率预测一首歌曲在Billboard上的成功。最成功的算法是随机森林和支持向量机。

摘要：In this study, we approached the Hit Song Prediction problem, which aims to predict which songs will become Billboard hits. We gathered a dataset of nearly 18500 hit and non-hit songs and extracted their audio features using the Spotify Web API. We test four machine-learning models on our dataset. We were able to predict the Billboard success of a song with approximately 86\% accuracy. The most succesful algorithms were Random Forest and Support Vector Machine.

【3】 THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

标题：THLNet：用于单声道语音增强的两级异质轻量级网络

链接：https://arxiv.org/abs/2301.07939

作者：Feng Dang,Qi Hu,Pengyuan Zhang

机构：Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China, School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

摘要：在这篇论文中，我们提出了一个两阶段的异质轻量型网路来进行单耳语音增强。具体而言，我们设计了一个新的两阶段框架，由粗粒度的全频带掩模估计阶段和细粒度的低频细化阶段组成。代替使用手工设计的实值滤波器，我们使用一种新颖的可学习复值矩形带宽（LCRB）滤波器组作为紧凑特征的提取器。此外，考虑到所提出的两阶段任务各自的特点，我们使用了异构结构，即：一个U形子网作为CoarseNet的主干，一个单尺度子网作为FineNet的主干。我们在VoiceBank + DEMAND和DNS数据集上进行了实验来评估所提出的方法。实验结果表明，该方法在保持较小的模型规模和较低的计算复杂度的前提下，性能优于现有方法。

摘要：In this paper, we propose a two-stage heterogeneous lightweight network for monaural speech enhancement. Specifically, we design a novel two-stage framework consisting of a coarse-grained full-band mask estimation stage and a fine-grained low-frequency refinement stage. Instead of using a hand-designed real-valued filter, we use a novel learnable complex-valued rectangular bandwidth (LCRB) filter bank as an extractor of compact features. Furthermore, considering the respective characteristics of the proposed two-stage task, we used a heterogeneous structure, i.e., a U-shaped subnetwork as the backbone of CoarseNet and a single-scale subnetwork as the backbone of FineNet. We conducted experiments on the VoiceBank + DEMAND and DNS datasets to evaluate the proposed approach. The experimental results show that the proposed method outperforms the current state-of-the-art methods, while maintaining relatively small model size and low computational complexity.

【4】 From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition

标题：从英语到更多语言：跨语言语音识别的参数高效模型重编程

链接：https://arxiv.org/abs/2301.07851

作者：Chao-Han Huck Yang,Bo Li,Yu Zhang,Nanxin Chen,Rohit Prabhavalkar,Tara N. Sainath,Trevor Strohman

机构：Google, USA, Georgia Institute of Technology, USA

备注：Submitted to ICASSP 2023. The project was initiated in May 2022 during a research internship at Google Research

摘要：本文提出了一种新的参数有效的基于神经模型重编程的跨语言语音识别学习框架，它可以将训练良好的英语自动语音识别（ASR）模型用于其他语言的识别。我们设计了不同的辅助神经架构，重点放在可学习的预训练特征增强，这是第一次授权模型重编程的ASR。具体而言，我们研究如何选择可训练组件（即，编码器）作为冻结的预训练骨干。在七种语言的多语言LibriSpeech语音（MLS）任务上的实验表明，模型重编程仅需要来自完整ASR模型的其原始可训练参数的4.2%（270 M中的11 M）到6.8%（660 M中的45 M），以执行在不同语言上平均的11.9%到8.1%WER范围内的竞争性结果。此外，我们发现了不同的设置，使大规模的预先训练的ASR成功地在单语和多语语音识别。我们的方法优于现有ASR调整架构及其具有自监督损耗的扩展（例如，w2 v-bert）在较低的WER和较好的训练效率方面。

摘要：In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages. We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement that, for the first time, empowers model reprogramming on ASR. Specifically, we investigate how to select trainable components (i.e., encoder) of a conformer-based RNN-Transducer, as a frozen pre-trained backbone. Experiments on a seven-language multilingual LibriSpeech speech (MLS) task show that model reprogramming only requires 4.2% (11M out of 270M) to 6.8% (45M out of 660M) of its original trainable parameters from a full ASR model to perform competitive results in a range of 11.9% to 8.1% WER averaged across different languages. In addition, we discover different setups to make large-scale pre-trained ASR succeed in both monolingual and multilingual speech recognition. Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses (e.g., w2v-bert) in terms of lower WER and better training efficiency.

【5】 Warning: Humans Cannot Reliably Detect Speech Deepfakes

标题：警告：人类无法可靠地检测到语音深伪

链接：https://arxiv.org/abs/2301.07829

作者：Kimberly T. Mai,Sergi D. Bray,Toby Davies,Lewis D. Griffin

机构：Department of Security and Crime Science, University College London, Department of Computer Science, University College London, Author Note, We gratefully acknowledge funding from the Dawes Centre for Future Crime and EPSRC, survey materials.

摘要：语音deepfakes是由机器学习模型生成的人工语音。之前的文献强调deepfakes是人工智能进步带来的最大安全威胁之一，因为它们有可能被滥用。然而，对人体检测能力的研究是有限的。我们向$n$ = 529个人提供了正版和deepfake音频，并要求他们识别deepfake。我们用英语和普通话进行了实验，以了解语言是否影响检测性能和决策原理。检测能力不可靠。听众正确识别deepfakes的概率只有73%，两种语言的可检测性没有差别。通过提供语音deepfakes的例子来提高听众的意识只会稍微改善结果。检测语音deepfakes的困难证实了它们被滥用的可能性，并表明需要防御这种威胁。

摘要：Speech deepfakes are artificial voices generated by machine learning models. Previous literature has highlighted deepfakes as one of the biggest threats to security arising from progress in AI due to their potential for misuse. However, studies investigating human detection capabilities are limited. We presented genuine and deepfake audio to $n$ = 529 individuals and asked them to identify the deepfakes. We ran our experiments in English and Mandarin to understand if language affects detection performance and decision-making rationale. Detection capability is unreliable. Listeners only correctly spotted the deepfakes 73% of the time, and there was no difference in detectability between the two languages. Increasing listener awareness by providing examples of speech deepfakes only improves results slightly. The difficulty of detecting speech deepfakes confirms their potential for misuse and signals that defenses against this threat are needed.

eess.AS音频处理