今天跟大家分享一篇语音相关的论文合集：cs.SD语音3篇，eess.AS音频处理3篇。


cs.SD语音

【1】 musicaiz: A Python Library for Symbolic Music Generation, Analysis and Visualization

标题：Musicaiz：一个用于符号音乐生成、分析和可视化的Python库

链接：https://arxiv.org/abs/2209.07974

作者：Carlos Hernandez-Olivan,Jose R. Beltran
机构：Department of Electronic Engineering and Communications, University of Zaragoza, Zaragoza, Spain
摘要：在这篇文章中，我们提出了一个面向对象的库musicaiz，用于分析、生成和评价符号音乐。该软件包的子模块允许用户从头开始创建符号音乐数据，构建算法来分析符号音乐，将MIDI数据编码为令牌来训练深度学习序列模型，修改现有音乐数据并评估音乐生成系统。评估子模块建立在先前工作的基础上，以客观地测量音乐生成系统，并且能够再现音乐生成模型的结果。该库可在网上公开获取。我们鼓励机构群体参与并提供反馈。
摘要：In this article, we present musicaiz, an object-oriented library for analyzing, generating and evaluating symbolic music. The submodules of the package allow the user to create symbolic music data from scratch, build algorithms to analyze symbolic music, encode MIDI data as tokens to train deep learning sequence models, modify existing music data and evaluate music generation systems. The evaluation submodule builds on previous work to objectively measure music generation systems and to be able to reproduce the results of music generation models. The library is publicly available online. We encourage the community to contribute and provide feedback.


【2】 Self-Relation Attention and Temporal Awareness for Emotion Recognition via Vocal Burst

标题：发声情绪再认的自我关系注意和时间知觉

链接：https://arxiv.org/abs/2209.07629

作者：Dang-Linh Trinh,Minh-Cong Vo,Guee-Sang Lee
机构：Department of AI Convergence, Chonnam National University), Gwangju, Korea
摘要：技术报告介绍了我们在ACII情感爆发（A-VB）2022工作坊\&竞赛中针对高维情感任务（A-VB High）的情感识别管道。我们提出的方法包括三个阶段。首先，通过自监督学习方法从原始音频信号及其Mel谱图中提取潜在特征。然后，将原始信号的特征馈送到自我关系注意和时间意识（SA-TA）模块，以学习这些潜在特征之间的有价值的信息。最后，我们连接所有的特征，并利用一个完全连接的层来预测每种情感的得分。通过实验，我们提出的方法在测试集上获得了0.7295的平均一致性相关系数（CCC），而在基线模型上为0.5686。我们的方法代码可在www.example.com上获得https://github.com/linhtd812/A-VB2022。
摘要：The technical report presents our emotion recognition pipeline for high-dimensional emotion task (A-VB High) in The ACII Affective Vocal Bursts (A-VB) 2022 Workshop \& Competition. Our proposed method contains three stages. Firstly, we extract the latent features from the raw audio signal and its Mel-spectrogram by self-supervised learning methods. Then, the features from the raw signal are fed to the self-relation attention and temporal awareness (SA-TA) module for learning the valuable information between these latent features. Finally, we concatenate all the features and utilize a fully-connected layer to predict each emotion's score. By empirical experiments, our proposed method achieves a mean concordance correlation coefficient (CCC) of 0.7295 on the test set, compared to 0.5686 on the baseline model. The code of our method is available at https://github.com/linhtd812/A-VB2022.


【3】 ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech

标题：ZeroEGGS：基于Zero-Shot实例的语音手势生成

链接：https://arxiv.org/abs/2209.07556

作者：Saeed Ghorbani,Ylva Ferstl,Daniel Holden,Nikolaus F. Troje,Marc-André Carbonneau
机构：Canada 2York University
摘要：通过实例介绍了一种基于ZeroEGGS神经网络的语音驱动手势生成方法。这意味着仅通过一个简短的示例运动剪辑就可以控制样式，即使是在训练过程中看不到的运动样式。我们的模型使用一个Variational框架来学习样式嵌入，使得通过潜在的空间操作或样式嵌入的混合和缩放来修改样式变得容易。我们的框架的概率性质进一步使得能够在给定相同输入的情况下生成各种输出，解决了手势运动的随机性质。在一系列的实验中，我们首先证明了我们的模型对新说话者和新风格的灵活性和普遍性。在用户研究中，我们证明了我们的模型在运动的自然性、语音的适当性和风格描绘方面优于以前的最先进的技术。最后，我们发布了一个高质量的全身手势运动数据集，包括手指和语音，跨越19种不同的风格。
摘要：We present ZeroEGGS, a neural network framework for speech-driven gesture generation with zero-shot style control by example. This means style can be controlled via only a short example motion clip, even for motion styles unseen during training. Our model uses a Variational framework to learn a style embedding, making it easy to modify style through latent space manipulation or blending and scaling of style embeddings. The probabilistic nature of our framework further enables the generation of a variety of outputs given the same input, addressing the stochastic nature of gesture motion. In a series of experiments, we first demonstrate the flexibility and generalizability of our model to new speakers and styles. In a user study, we then show that our model outperforms previous state-of-the-art techniques in naturalness of motion, appropriateness for speech, and style portrayal. Finally, we release a high-quality dataset of full-body gesture motion including fingers, with speech, spanning across 19 different styles.


eess.AS音频处理

【1】 Open Set Recognition For Music Genre Classification

标题：面向音乐流派分类的开集识别

链接：https://arxiv.org/abs/2209.07548

作者：Kevin Liu,Julien DeMori,Kobi Abayomi
机构：Seton Hall University, Moin Haque, Warner Music Group
备注：9 pages, 5 figures, 4 tables
摘要：我们使用开源的GTZAN和FMA数据集来探索已知和未知体裁类的分割。对于每一个，我们首先从最好的闭集体裁分类开始，然后我们应用开集识别方法。本文提出了一种基于OSR的音乐流派分类算法。我们展示了检索已知体裁的能力，以及识别新体裁的听觉模式（不出现在训练集中）。我们使用GTZAN和FMA数据集进行了四个实验，每个实验包含不同的已知和未知类别集，以建立新体裁检测的基线容量。我们在OpenMax和softmax上使用网格搜索来确定每个实验设置的最佳总分类准确率，并说明了体裁标记和开放集识别准确率之间的交互作用。
摘要：We explore segmentation of known and unknown genre classes using the open source GTZAN and FMA datasets. For each, we begin with best-case closed set genre classification, then we apply open set recognition methods. We offer an algorithm for the music genre classification task using OSR. We demonstrate the ability to retrieve known genres and as well identification of aural patterns for novel genres (not appearing in a training set). We conduct four experiments, each containing a different set of known and unknown classes, using the GTZAN and the FMA datasets to establish a baseline capacity for novel genre detection. We employ grid search on both OpenMax and softmax to determine the optimal total classification accuracy for each experimental setup, and illustrate interaction between genre labelling and open set recognition accuracy.


【2】 musicaiz: A Python Library for Symbolic Music Generation, Analysis and Visualization
标题：Musicaiz：一个用于符号音乐生成、分析和可视化的Python库
链接：https://arxiv.org/abs/2209.07974

* 与cs.SD语音【1】为同一篇

作者：Carlos Hernandez-Olivan,Jose R. Beltran
机构：Department of Electronic Engineering and Communications, University of Zaragoza, Zaragoza, Spain
摘要：在这篇文章中，我们提出了一个面向对象的库musicaiz，用于分析、生成和评价符号音乐。该软件包的子模块允许用户从头开始创建符号音乐数据，构建算法来分析符号音乐，将MIDI数据编码为令牌来训练深度学习序列模型，修改现有音乐数据并评估音乐生成系统。评估子模块建立在先前工作的基础上，以客观地测量音乐生成系统，并且能够再现音乐生成模型的结果。该库可在网上公开获取。我们鼓励机构群体参与并提供反馈。
摘要：In this article, we present musicaiz, an object-oriented library for analyzing, generating and evaluating symbolic music. The submodules of the package allow the user to create symbolic music data from scratch, build algorithms to analyze symbolic music, encode MIDI data as tokens to train deep learning sequence models, modify existing music data and evaluate music generation systems. The evaluation submodule builds on previous work to objectively measure music generation systems and to be able to reproduce the results of music generation models. The library is publicly available online. We encourage the community to contribute and provide feedback.


【3】 Self-Relation Attention and Temporal Awareness for Emotion Recognition via Vocal Burst
标题：发声情绪再认的自我关系注意和时间知觉
链接：https://arxiv.org/abs/2209.07629

* 与cs.SD语音【2】为同一篇

作者：Dang-Linh Trinh,Minh-Cong Vo,Guee-Sang Lee
机构：Department of AI Convergence, Chonnam National University), Gwangju, Korea
摘要：技术报告介绍了我们在ACII情感爆发（A-VB）2022工作坊\&竞赛中针对高维情感任务（A-VB High）的情感识别管道。我们提出的方法包括三个阶段。首先，通过自监督学习方法从原始音频信号及其Mel谱图中提取潜在特征。然后，将原始信号的特征馈送到自我关系注意和时间意识（SA-TA）模块，以学习这些潜在特征之间的有价值的信息。最后，我们连接所有的特征，并利用一个完全连接的层来预测每种情感的得分。通过实验，我们提出的方法在测试集上获得了0.7295的平均一致性相关系数（CCC），而在基线模型上为0.5686。我们的方法代码可在www.example.com上获得https://github.com/linhtd812/A-VB2022。
摘要：The technical report presents our emotion recognition pipeline for high-dimensional emotion task (A-VB High) in The ACII Affective Vocal Bursts (A-VB) 2022 Workshop \& Competition. Our proposed method contains three stages. Firstly, we extract the latent features from the raw audio signal and its Mel-spectrogram by self-supervised learning methods. Then, the features from the raw signal are fed to the self-relation attention and temporal awareness (SA-TA) module for learning the valuable information between these latent features. Finally, we concatenate all the features and utilize a fully-connected layer to predict each emotion's score. By empirical experiments, our proposed method achieves a mean concordance correlation coefficient (CCC) of 0.7295 on the test set, compared to 0.5686 on the baseline model. The code of our method is available at https://github.com/linhtd812/A-VB2022.

机器翻译，仅供参考