今日论文合集：cs.SD语音7篇，eess.AS音频处理8篇。
cs.SD语音

【1】AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head标题：AudioGPT：理解和生成语音、音乐、声音和说话头链接：http://arxiv.org/pdf/2304.12995v1

作者：Rongjie Huang,Mingze Li,Dongchao Yang,Jiatong Shi,Xuankai Chang,Zhenhui Ye,Yuning Wu,Zhiqing Hong,Jiawei Huang,Jinglin Liu,Yi Ren,Zhou Zhao,Shinji Watanabe机构：Zhejiang University, Peking University, Carnegie Mellon University, Remin University of China摘要：大型语言模型（LLM）在各种领域和任务中表现出了卓越的能力，挑战了我们对学习和认知的理解。尽管最近取得了成功，但目前的LLM无法处理复杂的音频信息或进行语音对话（如Siri或Alexa）。在这项工作中，我们提出了一个名为AudioGPT的多模态AI系统，它补充了LLM（即，ChatGPT）与1）处理复杂音频信息并解决众多理解和生成任务的基础模型;以及2）输入 输出接口（ASR，TTS），以支持口语对话。随着越来越多的需求，以评估多模态LLM的人类意图理解和合作的基础模型，我们概述的原则和过程，并测试AudioGPT的一致性，能力和鲁棒性。实验结果证明了AudioGPT在解决人工智能任务方面的能力，包括语音，音乐，声音和多轮对话中的说话头部理解和生成，使人类能够以前所未有的轻松方式创建丰富多样的音频内容。我们的系统在 url{https：github.com AIGC-Audio AudioGPT}上公开。摘要：Large language models (LLMs) have exhibited remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. Despite the recent success, current LLMs are not capable of processing complex audio information or conducting spoken conversations (like Siri or Alexa). In this work, we propose a multi-modal AI system named AudioGPT, which complements LLMs (i.e., ChatGPT) with 1) foundation models to process complex audio information and solve numerous understanding and generation tasks; and 2) the input output interface (ASR, TTS) to support spoken dialogue. With an increasing demand to evaluate multi-modal LLMs of human intention understanding and cooperation with foundation models, we outline the principles and processes and test AudioGPT in terms of consistency, capability, and robustness. Experimental results demonstrate the capabilities of AudioGPT in solving AI tasks with speech, music, sound, and talking head understanding and generation in multi-round dialogues, which empower humans to create rich and diverse audio content with unprecedented ease. Our system is publicly available at url{https: github.com AIGC-Audio AudioGPT}.

【2】Room dimensions and absorption inference from room transfer function via machine learning标题：基于机器学习的房间传递函数的房间尺寸和吸收推断链接：http://arxiv.org/pdf/2304.12993v1

作者：Yuanxin Xia,Cheol-Ho Jeong机构：Acoustic Technology, DTU Electro, Technical university of Denmark, Ørsteds Pl. , Kongens Lyngby , Denmark备注：15 pages, 10 figures摘要：仅使用声学信号来推断现有房间的吸收配置可能具有挑战性。本研究提出了两种方法来估计房间尺寸和频率依赖的吸收系数使用房间传递函数。第一种方法是基于知识的方法，通过房间的阻尼共振频率来计算房间尺寸。第二种方法是机器学习方法，采用多任务卷积神经网络来推断房间尺寸和每个表面的频率相关吸收系数。该研究表明，精确的基于波的模拟数据可以用来训练神经网络的真实世界的测量，并证明了这种算法被用来估计房间声学模拟的边界输入数据的潜力。所提出的方法可以是一个有价值的工具，在声学改造或干预项目的房间声学模拟，因为它们能够推断房间的几何形状和吸收条件，合理的小数据要求。摘要：The inference of the absorption configuration of an existing room solely using acoustic signals can be challenging. This research presents two methods for estimating the room dimensions and frequency-dependent absorption coefficients using room transfer functions. The first method, a knowledge-based approach, calculates the room dimensions through damped resonant frequencies of the room. The second method, a machine learning approach, employs multi-task convolutional neural networks for inferring the room dimensions and frequency-dependent absorption coefficients of each surface. The study shows that accurate wave-based simulation data can be used to train neural networks for real-world measurements and demonstrates a potential for this algorithm to be used to estimate the boundary input data for room acoustic simulations. The proposed methods can be a valuable tool for room acoustic simulations during acoustic renovation or intervention projects, as they enable to infer the room geometry and absorption conditions with reasonably small data requirements.

【3】The ACCompanion: Combining Reactivity, Robustness, and Musical Expressivity in an Automatic Piano Accompanist标题：ACCompanion：在自动钢琴伴奏者中结合反应性、健壮性和音乐表现力链接：http://arxiv.org/pdf/2304.12939v1

作者：Carlos Cancino-Chacón,Silvan Peter,Patricia Hu,Emmanouil Karystinaios,Florian Henkel,Francesco Foscarin,Nimrod Varga,Gerhard Widmer机构：Institute of Computational Perception, Johannes Kepler University Linz, Austria, SiriusXM + Pandora, USA备注：Accepted for the Arts and Creativity track at the 32nd International Joint Conference on Artificial Intelligence (IJCAI-23)摘要：本文介绍了一个富有表现力的伴奏系统ACCompanion。类似于一个音乐家谁陪同独奏演奏给定的音乐作品，我们的系统可以产生一个类似人类的伴奏部分，按照独奏者的选择，在节奏，动态和清晰度。ACCompanion在符号域中工作，即，它需要一种能够产生和演奏MIDI数据的乐器，该MIDI数据对于每个演奏的音符具有明确编码的起始、偏移和音高。我们描述了这样一个系统的组成部分，从实时分数跟踪和预测表达性能的生成和在线适应人类球员的表达选择。根据我们在不同观众面前反复进行现场演示的经验，我们对将这些组件组合成一个高度反应性和精确的系统所面临的挑战进行了分析，同时仍然是一个可靠的音乐伙伴，对可能的表演错误具有鲁棒性，并对表达变化做出反应。摘要：This paper introduces the ACCompanion, an expressive accompaniment system. Similarly to a musician who accompanies a soloist playing a given musical piece, our system can produce a human-like rendition of the accompaniment part that follows the soloist's choices in terms of tempo, dynamics, and articulation. The ACCompanion works in the symbolic domain, i.e., it needs a musical instrument capable of producing and playing MIDI data, with explicitly encoded onset, offset, and pitch for each played note. We describe the components that go into such a system, from real-time score following and prediction to expressive performance generation and online adaptation to the expressive choices of the human player. Based on our experience with repeated live demonstrations in front of various audiences, we offer an analysis of the challenges of combining these components into a system that is highly reactive and precise, while still a reliable musical partner, robust to possible performance errors and responsive to expressive variations.

【4】Can Voice Assistants Sound Cute? Towards a Model of Kawaii Vocalics标题：语音助手听起来可爱吗？走向Kawaii元音的典范链接：http://arxiv.org/pdf/2304.12809v1

作者：Katie Seaborn,Somang Nam,Julia Keckeis,Tatsuya Itagaki机构：Tokyo Institute of Technology, Tokyo, Japan, Toronto Metropolitan University, Toronto, Canada, TU Berlin, Berlin, GermanyJournal-ref：In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 63, 1-7摘要：日本的“卡哇伊”概念或可爱、脆弱和 或魅力的表达是一种全球文化出口。工作已经探索了卡哇伊性作为一个设计特征和用户体验的因素，在视觉外观，非语言行为，机器人和虚拟角色的声音。在这项初步工作中，我们考虑是否可以通过探索语音助理语音的声音质量来进行语音卡哇伊，即，卡哇伊声乐从年龄包容模型的卡哇伊，我们运行了一个用户感知研究的卡哇伊的年轻和老年人听起来日本电脑的声音。我们发现，卡哇伊性与性别和年龄的感知交叉，即，性别模糊和少女化，以及VA特征，即，流畅和做作。我们提出了一个初步的模型kawaii声乐验证通过识别和研究的声音品质，认知评价，行为反应，和情感的报告。摘要：The Japanese notion of "kawaii" or expressions of cuteness, vulnerability, and or charm is a global cultural export. Work has explored kawaii-ness as a design feature and factor of user experience in the visual appearance, nonverbal behaviour, and sound of robots and virtual characters. In this initial work, we consider whether voices can be kawaii by exploring the vocal qualities of voice assistant speech, i.e., kawaii vocalics. Drawing from an age-inclusive model of kawaii, we ran a user perceptions study on the kawaii-ness of younger- and older-sounding Japanese computer voices. We found that kawaii-ness intersected with perceptions of gender and age, i.e., gender ambiguous and girlish, as well as VA features, i.e., fluency and artificiality. We propose an initial model of kawaii vocalics to be validated through the identification and study of vocal qualities, cognitive appraisals, behavioural responses, and affective reports.

【5】Adaptive Representations of Sound for Automatic Insect Recognition标题：用于昆虫自动识别的声音自适应表示法链接：http://arxiv.org/pdf/2304.12739v1

作者：Marius Faiß,Dan Stowell机构：Naturalis Biodiversity Center, Leiden, The Netherlands, Leiden University, Leiden, The Netherlands, Department of Cognitive Science and AI, Tilburg University, Tilburg, The Netherlands备注：35 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:2211.09503摘要：随着时间的推移，昆虫种群数量和生物多样性迅速下降，监测这些趋势对于有效实施保护措施变得越来越重要。但监测方法通常是侵入性的，时间和资源密集，并且容易出现各种偏差。许多昆虫物种产生特征性的声音，这些声音可以容易地被检测和记录，而不需要大的成本或努力。使用深度学习方法，可以自动检测和分类来自野外记录的昆虫声音，以监测生物多样性和物种分布范围。我们使用最近发表的昆虫声音（直翅目和蝉科）和机器学习方法的数据集，并评估其潜在的声学昆虫监测。我们比较了传统的基于频谱图的音频表示对叶，一个新的自适应和基于波形的前端的性能。通过在训练过程中调整其特征提取参数，LEAF实现了比Mel谱图前端更好的分类性能。这一结果对于未来实现用于自动昆虫声音识别的深度学习技术是令人鼓舞的，特别是随着更大的数据集变得可用。摘要：Insect population numbers and biodiversity have been rapidly declining with time, and monitoring these trends has become increasingly important for conservation measures to be effectively implemented. But monitoring methods are often invasive, time and resource intense, and prone to various biases. Many insect species produce characteristic sounds that can easily be detected and recorded without large cost or effort. Using deep learning methods, insect sounds from field recordings could be automatically detected and classified to monitor biodiversity and species distribution ranges. We implement this using recently published datasets of insect sounds (Orthoptera and Cicadidae) and machine learning methods and evaluate their potential for acoustic insect monitoring. We compare the performance of the conventional spectrogram-based audio representation against LEAF, a new adaptive and waveform-based frontend. LEAF achieved better classification performance than the mel-spectrogram frontend by adapting its feature extraction parameters during training. This result is encouraging for future implementations of deep learning technology for automatic insect sound recognition, especially as larger datasets become available.

【6】GTN-Bailando: Genre Consistent Long-Term 3D Dance Generation based on Pre-trained Genre Token Network标题：GTN-Bailando：基于预先训练的流派表征网络的流派一致的长期3D舞蹈生成链接：http://arxiv.org/pdf/2304.12704v1

作者：Haolin Zhuang,Shun Lei,Long Xiao,Weiqin Li,Liyang Chen,Sicheng Yang,Zhiyong Wu,Shiyin Kang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, XVerse Inc., Shenzhen, China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：Accepted by ICASSP2023.Demo page: this https URL摘要：音乐驱动的3D舞蹈生成近年来已成为一个热门的研究课题，具有巨大的现实应用潜力。现有的方法大多缺乏对体裁的考虑，导致生成的舞蹈动作体裁不一致。此外，舞蹈类型与音乐之间的相关性尚未被研究。为了解决这些问题，我们提出了一个体裁一致的舞蹈生成框架，GTN-Bailando。首先，我们提出了类型令牌网络（GTN），它推断的音乐类型，以提高长期的舞蹈世代的类型的一致性。在AIST++数据集上的实验结果表明，本文提出的舞蹈生成框架在运动质量和风格一致性方面优于现有的方法。摘要：Music-driven 3D dance generation has become an intensive research topic in recent years with great potential for real-world applications. Most existing methods lack the consideration of genre, which results in genre inconsistency in the generated dance movements. In addition, the correlation between the dance genre and the music has not been investigated. To address these issues, we propose a genre-consistent dance generation framework, GTN-Bailando. First, we propose the Genre Token Network (GTN), which infers the genre from music to enhance the genre consistency of long-term dance generation. Second, to improve the generalization capability of the model, the strategy of pre-training and fine-tuning is adopted.Experimental results on the AIST++ dataset show that the proposed dance generation framework outperforms state-of-the-art methods in terms of motion quality and genre consistency.

【7】Foley Sound Synthesis at the DCASE 2023 Challenge标题：DCASE 2023挑战赛上的Foley声音合成链接：http://arxiv.org/pdf/2304.12521v1

作者：Keunwoo Choi,Jaekwon Im,Laurie Heller,Brian McFee,Keisuke Imoto,Yuki Okamoto,Mathieu Lagrange,Shinosuke Takamichi机构：KAIST, Daejeon, South Korea备注：DCASE 2023 Challenge - Task 7 - Technical Report摘要：在后期制作期间添加Foley声音效果是用于增强多媒体内容的感知声学特性的常用技术。传统上，Foley声音是由人类Foley艺术家制作的，涉及声音的手动录制和混合。然而，声音合成和生成模型的最新进展产生了机器辅助或自动Foley合成技术的兴趣。为了促进这一领域的进一步研究，我们在DCASE 2023中组织了一个挑战：任务7 - Foley声音合成。我们的挑战旨在提供一个标准化的评价框架，既严格和有效的，允许不同的福利合成系统的评价。通过这项挑战，我们希望鼓励研究界的积极参与，并推进自动Foley合成的最新技术。在本技术报告中，我们提供了Foley声音合成挑战的详细概述，包括任务定义，数据集，基线，评估方案和标准以及讨论。摘要：The addition of Foley sound effects during post-production is a common technique used to enhance the perceived acoustic properties of multimedia content. Traditionally, Foley sound has been produced by human Foley artists, which involves manual recording and mixing of sound. However, recent advances in sound synthesis and generative models have generated interest in machine-assisted or automatic Foley synthesis techniques. To promote further research in this area, we have organized a challenge in DCASE 2023: Task 7 - Foley Sound Synthesis. Our challenge aims to provide a standardized evaluation framework that is both rigorous and efficient, allowing for the evaluation of different Foley synthesis systems. Through this challenge, we hope to encourage active participation from the research community and advance the state-of-the-art in automatic Foley synthesis. In this technical report, we provide a detailed overview of the Foley sound synthesis challenge, including task definition, dataset, baseline, evaluation scheme and criteria, and discussion.
eess.AS音频处理

【1】Leveraging Audio-Tagging Assisted Sound Event Detection using Weakified Strong Labels and Frequency Dynamic Convolutions标题：利用弱化强标签和频率动态卷积的音频标记辅助声音事件检测链接：http://arxiv.org/pdf/2304.12688v1

作者：Tanmay Khandelwal,Rohan Kumar Das,Andrew Koh,Eng Siong Chng机构：∗Fortemedia Singapore, Singapore †Nanyang Technological University, Singapore备注：Accepted for Publication in IEEE-Statistical Signal Processing (SSP) Workshop 2023摘要：在半监督学习（SSL）中，从小标记集和大未标记集的联合学习是一个活跃的研究课题。在本文中，我们提出了一种新的SSL方法的基础上，利用一个大的未标记的域集的两个阶段的框架。我们提出的框架的第1阶段侧重于音频标记（AT），这有助于在第2阶段的声音事件检测（SED）系统。AT系统利用被转换为弱预测的强标记集（称为弱化集、弱标记集和未标记集）来训练。然后，该AT系统在未标记的集合上进行推断以生成可靠的伪弱标记，该伪弱标记与强标记的集合和弱标记的集合一起用于以监督的方式在阶段2训练基于频率动态卷积递归神经网络的SED系统。我们的系统优于基线的45.5%，在复调声音检测分数的DESED真正的验证集。摘要：Jointly learning from a small labeled set and a larger unlabeled set is an active research topic under semi-supervised learning (SSL). In this paper, we propose a novel SSL method based on a two-stage framework for leveraging a large unlabeled in-domain set. Stage-1 of our proposed framework focuses on audio-tagging (AT), which assists the sound event detection (SED) system in Stage-2. The AT system is trained utilizing a strongly labeled set converted into weak predictions referred to as weakified set, a weakly labeled set, and an unlabeled set. This AT system then infers on the unlabeled set to generate reliable pseudo-weak labels, which are used with the strongly and weakly labeled set to train a frequency dynamic convolutional recurrent neural network-based SED system at Stage-2 in a supervised manner. Our system outperforms the baseline by 45.5% in terms of polyphonic sound detection score on the DESED real validation set.

【2】Improving Speech Translation Accuracy and Time Efficiency with Fine-tuned wav2vec 2.0-based Speech Segmentation标题：基于Wave2vec 2.0的精调语音分割技术提高语音翻译准确率和时间效率链接：http://arxiv.org/pdf/2304.12659v1

作者：Ryo Fukuda,Katsuhito Sudoh,Satoshi Nakamura机构：Katsuhito Sudoh is with the Graduate School of Science and Technology, Nara Institute of Science and Technology备注：Submitted to IEEEACM Transactions on Audio, Speech, and Language Processing摘要：语音翻译（ST）自动地将源语言的话语转换成另一种语言的文本。将连续语音分割成较短的片段，称为语音分割，在ST中起着重要的作用。最近的分割方法训练模仿ST语料库的分割已经超过了传统的方法。Tsiamas等人提出了一种基于预先训练的语音编码器wav 2 vec 2.0的分割帧分类器（SFC）。他们的方法名为SHAS，保留了ST语料库分割的95-98%的BLEU分数。然而，由SHAS生成的段与ST语料库分割有很大不同，并且在多个组合话语的情况下往往更长。这是由于SHAS依赖于长度试探法，即，它将语音分割成易于翻译长度的片段，而没有充分考虑通过将它们分割成更短的片段来改善ST的可能性。较长的段通常会降低翻译质量和ST的时间效率。在这项研究中，我们扩展SHAS，以提高ST翻译的准确性和效率，分裂成更短的部分，对应于句子的语音。我们引入了一个简单的分割算法，使用SFC预测的移动平均值，而不依赖于长度启发式，并探索了wav 2 vec 2.0微调，以改进语音分割预测。我们的实验结果表明，我们的语音分割方法显着提高了语音翻译的质量和时间效率相比，SHAS。摘要：Speech translation (ST) automatically converts utterances in a source language into text in another language. Splitting continuous speech into shorter segments, known as speech segmentation, plays an important role in ST. Recent segmentation methods trained to mimic the segmentation of ST corpora have surpassed traditional approaches. Tsiamas et al. proposed a segmentation frame classifier (SFC) based on a pre-trained speech encoder called wav2vec 2.0. Their method, named SHAS, retains 95-98% of the BLEU score for ST corpus segmentation. However, the segments generated by SHAS are very different from ST corpus segmentation and tend to be longer with multiple combined utterances. This is due to SHAS's reliance on length heuristics, i.e., it splits speech into segments of easily translatable length without fully considering the potential for ST improvement by splitting them into even shorter segments. Longer segments often degrade translation quality and ST's time efficiency. In this study, we extended SHAS to improve ST translation accuracy and efficiency by splitting speech into shorter segments that correspond to sentences. We introduced a simple segmentation algorithm using the moving average of SFC predictions without relying on length heuristics and explored wav2vec 2.0 fine-tuning for improved speech segmentation prediction. Our experimental results reveal that our speech segmentation method significantly improved the quality and the time efficiency of speech translation compared to SHAS.

【3】AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head标题：AudioGPT：理解和生成语音、音乐、声音和说话头链接：http://arxiv.org/pdf/2304.12995v1

*与cs.SD语音【1】为同一篇

作者：Rongjie Huang,Mingze Li,Dongchao Yang,Jiatong Shi,Xuankai Chang,Zhenhui Ye,Yuning Wu,Zhiqing Hong,Jiawei Huang,Jinglin Liu,Yi Ren,Zhou Zhao,Shinji Watanabe机构：Zhejiang University, Peking University, Carnegie Mellon University, Remin University of China摘要：大型语言模型（LLM）在各种领域和任务中表现出了卓越的能力，挑战了我们对学习和认知的理解。尽管最近取得了成功，但目前的LLM无法处理复杂的音频信息或进行语音对话（如Siri或Alexa）。在这项工作中，我们提出了一个名为AudioGPT的多模态AI系统，它补充了LLM（即，ChatGPT）与1）处理复杂音频信息并解决众多理解和生成任务的基础模型;以及2）输入 输出接口（ASR，TTS），以支持口语对话。随着越来越多的需求，以评估多模态LLM的人类意图理解和合作的基础模型，我们概述的原则和过程，并测试AudioGPT的一致性，能力和鲁棒性。实验结果证明了AudioGPT在解决人工智能任务方面的能力，包括语音，音乐，声音和多轮对话中的说话头部理解和生成，使人类能够以前所未有的轻松方式创建丰富多样的音频内容。我们的系统在 url{https：github.com AIGC-Audio AudioGPT}上公开。摘要：Large language models (LLMs) have exhibited remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. Despite the recent success, current LLMs are not capable of processing complex audio information or conducting spoken conversations (like Siri or Alexa). In this work, we propose a multi-modal AI system named AudioGPT, which complements LLMs (i.e., ChatGPT) with 1) foundation models to process complex audio information and solve numerous understanding and generation tasks; and 2) the input output interface (ASR, TTS) to support spoken dialogue. With an increasing demand to evaluate multi-modal LLMs of human intention understanding and cooperation with foundation models, we outline the principles and processes and test AudioGPT in terms of consistency, capability, and robustness. Experimental results demonstrate the capabilities of AudioGPT in solving AI tasks with speech, music, sound, and talking head understanding and generation in multi-round dialogues, which empower humans to create rich and diverse audio content with unprecedented ease. Our system is publicly available at url{https: github.com AIGC-Audio AudioGPT}.

【4】Room dimensions and absorption inference from room transfer function via machine learning标题：基于机器学习的房间传递函数的房间尺寸和吸收推断链接：http://arxiv.org/pdf/2304.12993v1

*与cs.SD语音【2】为同一篇

作者：Yuanxin Xia,Cheol-Ho Jeong机构：Acoustic Technology, DTU Electro, Technical university of Denmark, Ørsteds Pl. , Kongens Lyngby , Denmark备注：15 pages, 10 figures摘要：仅使用声学信号来推断现有房间的吸收配置可能具有挑战性。本研究提出了两种方法来估计房间尺寸和频率依赖的吸收系数使用房间传递函数。第一种方法是基于知识的方法，通过房间的阻尼共振频率来计算房间尺寸。第二种方法是机器学习方法，采用多任务卷积神经网络来推断房间尺寸和每个表面的频率相关吸收系数。该研究表明，精确的基于波的模拟数据可以用来训练神经网络的真实世界的测量，并证明了这种算法被用来估计房间声学模拟的边界输入数据的潜力。所提出的方法可以是一个有价值的工具，在声学改造或干预项目的房间声学模拟，因为它们能够推断房间的几何形状和吸收条件，合理的小数据要求。摘要：The inference of the absorption configuration of an existing room solely using acoustic signals can be challenging. This research presents two methods for estimating the room dimensions and frequency-dependent absorption coefficients using room transfer functions. The first method, a knowledge-based approach, calculates the room dimensions through damped resonant frequencies of the room. The second method, a machine learning approach, employs multi-task convolutional neural networks for inferring the room dimensions and frequency-dependent absorption coefficients of each surface. The study shows that accurate wave-based simulation data can be used to train neural networks for real-world measurements and demonstrates a potential for this algorithm to be used to estimate the boundary input data for room acoustic simulations. The proposed methods can be a valuable tool for room acoustic simulations during acoustic renovation or intervention projects, as they enable to infer the room geometry and absorption conditions with reasonably small data requirements.

【5】The ACCompanion: Combining Reactivity, Robustness, and Musical Expressivity in an Automatic Piano Accompanist标题：ACCompanion：在自动钢琴伴奏者中结合反应性、健壮性和音乐表现力链接：http://arxiv.org/pdf/2304.12939v1

*与cs.SD语音【3】为同一篇

作者：Carlos Cancino-Chacón,Silvan Peter,Patricia Hu,Emmanouil Karystinaios,Florian Henkel,Francesco Foscarin,Nimrod Varga,Gerhard Widmer机构：Institute of Computational Perception, Johannes Kepler University Linz, Austria, SiriusXM + Pandora, USA备注：Accepted for the Arts and Creativity track at the 32nd International Joint Conference on Artificial Intelligence (IJCAI-23)摘要：本文介绍了一个富有表现力的伴奏系统ACCompanion。类似于一个音乐家谁陪同独奏演奏给定的音乐作品，我们的系统可以产生一个类似人类的伴奏部分，按照独奏者的选择，在节奏，动态和清晰度。ACCompanion在符号域中工作，即，它需要一种能够产生和演奏MIDI数据的乐器，该MIDI数据对于每个演奏的音符具有明确编码的起始、偏移和音高。我们描述了这样一个系统的组成部分，从实时分数跟踪和预测表达性能的生成和在线适应人类球员的表达选择。根据我们在不同观众面前反复进行现场演示的经验，我们对将这些组件组合成一个高度反应性和精确的系统所面临的挑战进行了分析，同时仍然是一个可靠的音乐伙伴，对可能的表演错误具有鲁棒性，并对表达变化做出反应。摘要：This paper introduces the ACCompanion, an expressive accompaniment system. Similarly to a musician who accompanies a soloist playing a given musical piece, our system can produce a human-like rendition of the accompaniment part that follows the soloist's choices in terms of tempo, dynamics, and articulation. The ACCompanion works in the symbolic domain, i.e., it needs a musical instrument capable of producing and playing MIDI data, with explicitly encoded onset, offset, and pitch for each played note. We describe the components that go into such a system, from real-time score following and prediction to expressive performance generation and online adaptation to the expressive choices of the human player. Based on our experience with repeated live demonstrations in front of various audiences, we offer an analysis of the challenges of combining these components into a system that is highly reactive and precise, while still a reliable musical partner, robust to possible performance errors and responsive to expressive variations.

【6】Adaptive Representations of Sound for Automatic Insect Recognition标题：用于昆虫自动识别的声音自适应表示法链接：http://arxiv.org/pdf/2304.12739v1

*与cs.SD语音【5】为同一篇

作者：Marius Faiß,Dan Stowell机构：Naturalis Biodiversity Center, Leiden, The Netherlands, Leiden University, Leiden, The Netherlands, Department of Cognitive Science and AI, Tilburg University, Tilburg, The Netherlands备注：35 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:2211.09503摘要：随着时间的推移，昆虫种群数量和生物多样性迅速下降，监测这些趋势对于有效实施保护措施变得越来越重要。但监测方法通常是侵入性的，时间和资源密集，并且容易出现各种偏差。许多昆虫物种产生特征性的声音，这些声音可以容易地被检测和记录，而不需要大的成本或努力。使用深度学习方法，可以自动检测和分类来自野外记录的昆虫声音，以监测生物多样性和物种分布范围。我们使用最近发表的昆虫声音（直翅目和蝉科）和机器学习方法的数据集，并评估其潜在的声学昆虫监测。我们比较了传统的基于频谱图的音频表示对叶，一个新的自适应和基于波形的前端的性能。通过在训练过程中调整其特征提取参数，LEAF实现了比Mel谱图前端更好的分类性能。这一结果对于未来实现用于自动昆虫声音识别的深度学习技术是令人鼓舞的，特别是随着更大的数据集变得可用。摘要：Insect population numbers and biodiversity have been rapidly declining with time, and monitoring these trends has become increasingly important for conservation measures to be effectively implemented. But monitoring methods are often invasive, time and resource intense, and prone to various biases. Many insect species produce characteristic sounds that can easily be detected and recorded without large cost or effort. Using deep learning methods, insect sounds from field recordings could be automatically detected and classified to monitor biodiversity and species distribution ranges. We implement this using recently published datasets of insect sounds (Orthoptera and Cicadidae) and machine learning methods and evaluate their potential for acoustic insect monitoring. We compare the performance of the conventional spectrogram-based audio representation against LEAF, a new adaptive and waveform-based frontend. LEAF achieved better classification performance than the mel-spectrogram frontend by adapting its feature extraction parameters during training. This result is encouraging for future implementations of deep learning technology for automatic insect sound recognition, especially as larger datasets become available.

【7】GTN-Bailando: Genre Consistent Long-Term 3D Dance Generation based on Pre-trained Genre Token Network标题：GTN-Bailando：基于预先训练的流派表征网络的流派一致的长期3D舞蹈生成链接：http://arxiv.org/pdf/2304.12704v1

*与cs.SD语音【6】为同一篇

作者：Haolin Zhuang,Shun Lei,Long Xiao,Weiqin Li,Liyang Chen,Sicheng Yang,Zhiyong Wu,Shiyin Kang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, XVerse Inc., Shenzhen, China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：Accepted by ICASSP2023.Demo page: this https URL摘要：音乐驱动的3D舞蹈生成近年来已成为一个热门的研究课题，具有巨大的现实应用潜力。现有的方法大多缺乏对体裁的考虑，导致生成的舞蹈动作体裁不一致。此外，舞蹈类型与音乐之间的相关性尚未被研究。为了解决这些问题，我们提出了一个体裁一致的舞蹈生成框架，GTN-Bailando。首先，我们提出了类型令牌网络（GTN），它推断的音乐类型，以提高长期的舞蹈世代的类型的一致性。在AIST++数据集上的实验结果表明，本文提出的舞蹈生成框架在运动质量和风格一致性方面优于现有的方法。摘要：Music-driven 3D dance generation has become an intensive research topic in recent years with great potential for real-world applications. Most existing methods lack the consideration of genre, which results in genre inconsistency in the generated dance movements. In addition, the correlation between the dance genre and the music has not been investigated. To address these issues, we propose a genre-consistent dance generation framework, GTN-Bailando. First, we propose the Genre Token Network (GTN), which infers the genre from music to enhance the genre consistency of long-term dance generation. Second, to improve the generalization capability of the model, the strategy of pre-training and fine-tuning is adopted.Experimental results on the AIST++ dataset show that the proposed dance generation framework outperforms state-of-the-art methods in terms of motion quality and genre consistency.

【8】Foley Sound Synthesis at the DCASE 2023 Challenge标题：DCASE 2023挑战赛上的Foley声音合成链接：http://arxiv.org/pdf/2304.12521v1

*与cs.SD语音【7】为同一篇

作者：Keunwoo Choi,Jaekwon Im,Laurie Heller,Brian McFee,Keisuke Imoto,Yuki Okamoto,Mathieu Lagrange,Shinosuke Takamichi机构：KAIST, Daejeon, South Korea备注：DCASE 2023 Challenge - Task 7 - Technical Report摘要：在后期制作期间添加Foley声音效果是用于增强多媒体内容的感知声学特性的常用技术。传统上，Foley声音是由人类Foley艺术家制作的，涉及声音的手动录制和混合。然而，声音合成和生成模型的最新进展产生了机器辅助或自动Foley合成技术的兴趣。为了促进这一领域的进一步研究，我们在DCASE 2023中组织了一个挑战：任务7 - Foley声音合成。我们的挑战旨在提供一个标准化的评价框架，既严格和有效的，允许不同的福利合成系统的评价。通过这项挑战，我们希望鼓励研究界的积极参与，并推进自动Foley合成的最新技术。在本技术报告中，我们提供了Foley声音合成挑战的详细概述，包括任务定义，数据集，基线，评估方案和标准以及讨论。摘要：The addition of Foley sound effects during post-production is a common technique used to enhance the perceived acoustic properties of multimedia content. Traditionally, Foley sound has been produced by human Foley artists, which involves manual recording and mixing of sound. However, recent advances in sound synthesis and generative models have generated interest in machine-assisted or automatic Foley synthesis techniques. To promote further research in this area, we have organized a challenge in DCASE 2023: Task 7 - Foley Sound Synthesis. Our challenge aims to provide a standardized evaluation framework that is both rigorous and efficient, allowing for the evaluation of different Foley synthesis systems. Through this challenge, we hope to encourage active participation from the research community and advance the state-of-the-art in automatic Foley synthesis. In this technical report, we provide a detailed overview of the Foley sound synthesis challenge, including task definition, dataset, baseline, evaluation scheme and criteria, and discussion.

机器翻译，仅供参考