今日论文合集：cs.SD语音12篇，eess.AS音频处理12篇。cs.SD语音【1】 Basic syntax from speech: Spontaneous concatenation in unsupervised deep neural networks标题：来自语音的基本句法：无监督深度神经网络中的自发连接链接：https://arxiv.org/abs/2305.01626作者：Gašper Beguš,Thomas Lu,Zili Wang机构：UC Berkeley摘要：句法的计算模型主要是基于文本的。在这里，我们建议，基本的语法可以直接从原始语音建模在一个完全无监督的方式。我们专注于语法中最普遍和最基本的属性之一--连接。我们引入自发级联：这是一种现象，在单个单词的声学记录上训练的卷积神经网络（CNN）开始生成具有两个甚至三个单词的输出，而无需访问输入中具有多个单词的数据。此外，在两个单词上训练的网络学习将单词嵌入到新的未观察到的单词组合中。据我们所知，这是在生成对抗网络设置中对原始语音进行训练的CNN的一个以前未报告的属性，并且对于我们理解这些架构如何学习以及建模语法及其从原始声学输入的演变都有影响。摘要：Computational models of syntax are predominantly text-based. Here we propose that basic syntax can be modeled directly from raw speech in a fully unsupervised way. We focus on one of the most ubiquitous and basic properties of syntax -- concatenation. We introduce spontaneous concatenation: a phenomenon where convolutional neural networks (CNNs) trained on acoustic recordings of individual words start generating outputs with two or even three words concatenated without ever accessing data with multiple words in the input. Additionally, networks trained on two words learn to embed words into novel unobserved word combinations. To our knowledge, this is a previously unreported property of CNNs trained on raw speech in the Generative Adversarial Network setting and has implications both for our understanding of how these architectures learn as well as for modeling syntax and its evolution from raw acoustic inputs.【2】 A Study on the Integration of Pipeline and E2E SLU systems for Spoken Semantic Parsing toward STOP Quality Challenge标题：面向Stop Quality挑战的口语语义分析流水线与E2E SLU系统集成研究链接：https://arxiv.org/abs/2305.01620作者：Siddhant Arora,Hayato Futami,Shih-Lun Wu,Jessica Huynh,Yifan Peng,Yosuke Kashiwagi,Emiru Tsunoo,Brian Yan,Shinji Watanabe机构：Carnegie Mellon University,Sony Group Corporation, Japan备注：First Place in Track 1 of STOP Challenge, which is part of ICASSP Signal Processing Grand Challenge 2023摘要：最近，人们一直在努力为口语理解（SLU）引入新的基准任务，如语义解析。在本文中，我们描述了我们提出的口语语义解析系统的质量轨道（轨道1）在口语理解大挑战，这是ICASSP信号处理大挑战2023的一部分。我们尝试使用端到端和管道系统来执行此任务。强大的自动语音识别（ASR）模型（如Whisper）和预训练的语言模型（LM）（如BART）在我们的SLU框架中使用，以提高性能。我们还研究了各种模型的输出水平组合，以获得80的精确匹配精度。8，在挑战赛中获得第一名。摘要：Recently there have been efforts to introduce new benchmark tasks for spoken language understanding (SLU), like semantic parsing. In this paper, we describe our proposed spoken semantic parsing system for the quality track (Track 1) in Spoken Language Understanding Grand Challenge which is part of ICASSP Signal Processing Grand Challenge 2023. We experiment with both end-to-end and pipeline systems for this task. Strong automatic speech recognition (ASR) models like Whisper and pretrained Language models (LM) like BART are utilized inside our SLU framework to boost performance. We also investigate the output level combination of various models to get an exact match accuracy of 80.8, which won the 1st place at the challenge.【3】 Self-supervised learning for infant cry analysis标题：用于婴儿哭闹分析的自我监督学习链接：https://arxiv.org/abs/2305.01578作者：Arsenii Gorin,Cem Subakan,Sajjad Abdoli,Junhao Wang,Samantha Latremouille,Charles Onu机构：⋆Ubenwa Health, Montr´eal, Canada, ♮Universit´e Laval, Qu´ebec City, Canada, ♯Concordia University, Montr´eal, Canada, ♭Mila-Qu´ebec AI Institute, Montr´eal, Canada备注：Accepted to IEEE ICASSP 2023 workshop Self-supervision in Audio, Speech and Beyond摘要：在本文中，我们探讨了自我监督学习（SSL）分析的第一个同类数据库的哭记录包含临床指征的1000多个新生儿。具体来说，我们的目标是基于哭泣的神经损伤检测以及识别哭泣触发器，如疼痛，饥饿和不适。在医疗环境中注释大型数据库是昂贵且耗时的，通常需要几位专家多年的合作。利用大量未标记的音频数据来学习有用的表示可以降低构建鲁棒模型的成本，并最终降低临床解决方案的成本。在这项工作中，我们在大型音频数据集上进行了卷积神经网络的自监督预训练实验。我们发现，SSL对比损失（SimCLR）的预训练在神经损伤和哭泣触发方面的表现明显优于监督预训练。此外，我们展示了进一步的性能增益，通过SSL为基础的域适应使用未标记的婴儿哭声。我们还表明，使用这种基于SSL的预训练适应哭泣的声音减少了对整个系统的标记数据的需要。摘要：In this paper, we explore self-supervised learning (SSL) for analyzing a first-of-its-kind database of cry recordings containing clinical indications of more than a thousand newborns. Specifically, we target cry-based detection of neurological injury as well as identification of cry triggers such as pain, hunger, and discomfort. Annotating a large database in the medical setting is expensive and time-consuming, typically requiring the collaboration of several experts over years. Leveraging large amounts of unlabeled audio data to learn useful representations can lower the cost of building robust models and, ultimately, clinical solutions. In this work, we experiment with self-supervised pre-training of a convolutional neural network on large audio datasets. We show that pre-training with SSL contrastive loss (SimCLR) performs significantly better than supervised pre-training for both neuro injury and cry triggers. In addition, we demonstrate further performance gains through SSL-based domain adaptation using unlabeled infant cries. We also show that using such SSL-based pre-training for adaptation to cry sounds decreases the need for labeled data of the overall system.【4】 Multitask learning in Audio Captioning: a sentence embedding regression loss acts as a regularizer标题：音频字幕中的多任务学习：嵌入回归损失的句子充当正则化链接：https://arxiv.org/abs/2305.01482作者：Etienne Labbé,Julien Pinquier,Thomas Pellegrini机构：IRIT, Universit´e de Toulouse, CNRS, UT, Toulouse, France摘要：在这项工作中，我们建议研究使用句子嵌入回归损失组件训练的模型在自动音频字幕任务中的性能。该任务旨在构建可以用自然语言编写的单个句子描述音频内容的系统。大多数系统都是用标准的交叉熵损失来训练的，它没有考虑句子的语义接近度。我们发现，添加一个句子嵌入损失项减少了过拟合，但也增加了SPIDER从0。397到0.418在我们的第一个设置上的AudioCaps语料库。当我们增加权重衰减值时，我们发现我们的模型更接近当前最先进的方法，SPIDEr得分高达0。444与0相比。475分。此外，该模型使用少八倍的可训练参数。在这种训练设置中，句子嵌入损失对模型性能没有更多的影响。摘要：In this work, we propose to study the performance of a model trained with a sentence embedding regression loss component for the Automated Audio Captioning task. This task aims to build systems that can describe audio content with a single sentence written in natural language. Most systems are trained with the standard Cross-Entropy loss, which does not take into account the semantic closeness of the sentence. We found that adding a sentence embedding loss term reduces overfitting, but also increased SPIDEr from 0.397 to 0.418 in our first setting on the AudioCaps corpus. When we increased the weight decay value, we found our model to be much closer to the current state- of-the-art methods, with a SPIDEr score up to 0.444 compared to a 0.475 score. Moreover, this model uses eight times less trainable parameters. In this training setting, the sentence embedding loss has no more impact on the model performance.【5】 Deep Learning Based Multimodal with Two-phase Training Strategy for Daily Life Video Classification标题：基于深度学习的多模式两阶段训练日常生活视频分类策略链接：https://arxiv.org/abs/2305.01476作者：Lam Pham,Trang Le,Cam Le,Dat Ngo,Weissenfeld Axel,Alexander Schindler机构：Le is with HCM University of Technology, Ngo is with School of Computer Science and Electronic Engineering, University of Essex摘要：在本文中，我们提出了一个基于深度学习的多模态系统，用于分类日常生活视频。为了训练系统，我们提出了一个两阶段的训练策略。在第一个训练阶段（第一阶段），我们从原始视频中提取音频和视觉（图像）数据。然后，我们使用独立的基于深度学习的模型来训练音频数据和视觉数据。在训练过程之后，我们通过从预训练的深度学习模型中提取特征图来获得音频嵌入和视觉嵌入。在第二个训练阶段（第二阶段），我们训练一个融合层来组合音频/视频嵌入，并训练一个密集层来将组合的嵌入分类到目标日常场景中。我们在DCASE（IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events）2021 Task 1B Development的基准数据集上进行了广泛的实验，达到了80的最佳分类准确率。5%、91. 8%和95。3%仅具有音频数据，仅具有视觉数据，分别具有音频和视觉数据。最高分类准确率达95.3%，改善了17。9%，与DCASE基线相比，显示出非常具有竞争力的国家的最先进的系统。摘要：In this paper, we present a deep learning based multimodal system for classifying daily life videos. To train the system, we propose a two-phase training strategy. In the first training phase (Phase I), we extract the audio and visual (image) data from the original video. We then train the audio data and the visual data with independent deep learning based models. After the training processes, we obtain audio embeddings and visual embeddings by extracting feature maps from the pre-trained deep learning models. In the second training phase (Phase II), we train a fusion layer to combine the audio/visual embeddings and a dense layer to classify the combined embedding into target daily scenes. Our extensive experiments, which were conducted on the benchmark dataset of DCASE (IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events) 2021 Task 1B Development, achieved the best classification accuracy of 80.5%, 91.8%, and 95.3% with only audio data, with only visual data, both audio and visual data, respectively. The highest classification accuracy of 95.3% presents an improvement of 17.9% compared with DCASE baseline and shows very competitive to the state-of-the-art systems.【6】 Long-Term Rhythmic Video Soundtracker标题：长期有节奏的视频音响追踪器链接：https://arxiv.org/abs/2305.01319作者：Jiashuo Yu,Yaohui Wang,Xinyuan Chen,Xiao Sun,Yu Qiao备注：ICML2023摘要：我们考虑的问题，产生的音乐音轨同步有节奏的视觉线索。大多数现有的作品依赖于预先定义的音乐表示，导致生成的灵活性和复杂性的无能。直接生成视频调节波形的其他方法遭受有限的场景、短长度和不稳定的生成质量。为此，我们提出了长期的节奏视频声音跟踪器（LORIS），一个新的框架来合成长期的条件波形。具体而言，我们的框架包括一个潜在的条件扩散概率模型来执行波形合成。此外，提出了一系列的上下文感知的条件编码器，考虑到时间信息的长期生成。值得注意的是，我们扩展我们的模型的适用性从舞蹈到多个运动场景，如自由体操和花样滑冰。为了进行全面的评估，我们建立了一个基准的节奏视频音轨，包括预处理的数据集，改进的评估指标，和强大的生成基线。大量的实验表明，我们的模型生成长期的音轨与国家的最先进的音乐质量和节奏的对应关系。代码可在\url{https：//github.com/OpenGVLab/LORIS}。摘要：We consider the problem of generating musical soundtracks in sync with rhythmic visual cues. Most existing works rely on pre-defined music representations, leading to the incompetence of generative flexibility and complexity. Other methods directly generating video-conditioned waveforms suffer from limited scenarios, short lengths, and unstable generation quality. To this end, we present Long-Term Rhythmic Video Soundtracker (LORIS), a novel framework to synthesize long-term conditional waveforms. Specifically, our framework consists of a latent conditional diffusion probabilistic model to perform waveform synthesis. Furthermore, a series of context-aware conditioning encoders are proposed to take temporal information into consideration for a long-term generation. Notably, we extend our model's applicability from dances to multiple sports scenarios such as floor exercise and figure skating. To perform comprehensive evaluations, we establish a benchmark for rhythmic video soundtracks including the pre-processed dataset, improved evaluation metrics, and robust generative baselines. Extensive experiments show that our model generates long-term soundtracks with state-of-the-art musical quality and rhythmic correspondence. Codes are available at \url{https://github.com/OpenGVLab/LORIS}.【7】 AQ-GT: a Temporally Aligned and Quantized GRU-Transformer for Co-Speech Gesture Synthesis标题：AQ-GT：一种用于协同语音手势合成的时间对准量化GRU转换器链接：https://arxiv.org/abs/2305.01241作者：Hendric Voß,Stefan Kopp机构：Social Cognitive Systems Group, Bielefeld University, Germany摘要：现实的和上下文相关的协同语音手势的生成是一个具有挑战性的，但越来越重要的任务，在创建多模态人工代理。先前的方法集中于学习共同语音手势表示和所产生的运动之间的直接对应关系，这在人类评估期间创建了看似自然但通常不令人信服的手势。我们提出了一种方法来预训练部分手势序列使用生成对抗网络与量化管道。由此产生的码本向量作为输入和输出，在我们的框架，形成的基础上生成和重建的手势。通过学习潜在空间表示的映射而不是直接将其映射到矢量表示，该框架有助于生成高度逼真和富有表现力的手势，这些手势紧密地复制人类运动和行为，同时避免生成过程中的伪影。我们评估我们的方法，通过比较它与已建立的方法生成语音手势，以及与现有的人类行为的数据集。我们还进行了消融研究，以评估我们的发现。结果表明，我们的方法优于目前的最先进的一个明确的保证金，是部分无法区分人类手势。我们公开了我们的数据管道和生成框架。摘要：The generation of realistic and contextually relevant co-speech gestures is a challenging yet increasingly important task in the creation of multimodal artificial agents. Prior methods focused on learning a direct correspondence between co-speech gesture representations and produced motions, which created seemingly natural but often unconvincing gestures during human assessment. We present an approach to pre-train partial gesture sequences using a generative adversarial network with a quantization pipeline. The resulting codebook vectors serve as both input and output in our framework, forming the basis for the generation and reconstruction of gestures. By learning the mapping of a latent space representation as opposed to directly mapping it to a vector representation, this framework facilitates the generation of highly realistic and expressive gestures that closely replicate human movement and behavior, while simultaneously avoiding artifacts in the generation process. We evaluate our approach by comparing it with established methods for generating co-speech gestures as well as with existing datasets of human behavior. We also perform an ablation study to assess our findings. The results show that our approach outperforms the current state of the art by a clear margin and is partially indistinguishable from human gesturing. We make our data pipeline and the generation framework publicly available.【8】 The Pipeline System of ASR and NLU with MLM-based Data Augmentation toward STOP Low-resource Challenge标题：基于MLM数据增强的ASR和NLU流水线系统应对低资源挑战链接：https://arxiv.org/abs/2305.01194作者：Hayato Futami,Jessica Huynh,Siddhant Arora,Shih-Lun Wu,Yosuke Kashiwagi,Yifan Peng,Brian Yan,Emiru Tsunoo,Shinji Watanabe机构：Sony Group Corporation, Japan ,Carnegie Mellon University, USA备注：To be appeared at ICASSP2023摘要：本文描述了我们的系统，用于口语理解大挑战中的低资源域适应轨道（轨道3），这是ICASSP信号处理大挑战2023的一部分。在赛道上，我们采用了ASR和NLU的管道方法。对于ASR，我们通过上采样对每个域的Whisper进行微调。对于NLU，我们在所有Track 3数据上微调BART，然后在低资源域数据上微调BART。我们应用基于掩码LM（MLM）的数据增强，其中一些输入标记和相应的目标标签使用MLM替换。我们还采用了基于检索的方法，其中模型输入增加了类似的训练样本。结果，我们实现了精确匹配（EM）精度63。3/75.0（平均值：69.15）为提醒/天气领域，并在挑战赛中获得第一名。摘要：This paper describes our system for the low-resource domain adaptation track (Track 3) in Spoken Language Understanding Grand Challenge, which is a part of ICASSP Signal Processing Grand Challenge 2023. In the track, we adopt a pipeline approach of ASR and NLU. For ASR, we fine-tune Whisper for each domain with upsampling. For NLU, we fine-tune BART on all the Track3 data and then on low-resource domain data. We apply masked LM (MLM) -based data augmentation, where some of input tokens and corresponding target labels are replaced using MLM. We also apply a retrieval-based approach, where model input is augmented with similar training samples. As a result, we achieved exact match (EM) accuracy 63.3/75.0 (average: 69.15) for reminder/weather domain, and won the 1st place at the challenge.【9】 Contrastive Speech Mixup for Low-resource Keyword Spotting标题：用于低资源关键词检测的对比语音混合链接：https://arxiv.org/abs/2305.01170作者：Dianwen Ng,Ruixi Zhang,Jia Qi Yip,Chong Zhang,Yukun Ma,Trung Hieu Nguyen,Chongjia Ni,Eng Siong Chng,Bin Ma机构：Alibaba Group, School of Computer Science and Engineering, Nanyang Technological University, Singapore备注：Accepted by ICASSP 2023摘要：智能设备中大多数现有的基于神经的关键词识别模型（KWS）需要数千个训练样本来学习一个像样的音频表示。然而，随着智能设备变得更加个性化的需求不断增长，KWS模型需要快速适应较小的用户样本。为了应对这一挑战，我们提出了一个对比语音混合（CosMix）学习算法的低资源KWS。CosMix引入了一个辅助对比损失现有的混合增强技术，以最大限度地提高原始预混合样本和增强样本之间的相对相似性。我们的目标是注入增强约束，以引导模型从两个增强视图（即：即噪声混合和干净的预混合话语）。我们在Google Speech Command数据集上进行实验，将训练集的大小调整为2。每个关键字5分钟，模拟低资源条件。我们的实验结果表明，在多个模型的性能，表现出我们的方法的有效性一致的改善。摘要：Most of the existing neural-based models for keyword spotting (KWS) in smart devices require thousands of training samples to learn a decent audio representation. However, with the rising demand for smart devices to become more personalized, KWS models need to adapt quickly to smaller user samples. To tackle this challenge, we propose a contrastive speech mixup (CosMix) learning algorithm for low-resource KWS. CosMix introduces an auxiliary contrastive loss to the existing mixup augmentation technique to maximize the relative similarity between the original pre-mixed samples and the augmented samples. The goal is to inject enhancing constraints to guide the model towards simpler but richer content-based speech representations from two augmented views (i.e. noisy mixed and clean pre-mixed utterances). We conduct our experiments on the Google Speech Command dataset, where we trim the size of the training set to as small as 2.5 mins per keyword to simulate a low-resource condition. Our experimental results show a consistent improvement in the performance of multiple models, which exhibits the effectiveness of our method.【10】 LooPy: A Research-Friendly Mix Framework for Music Information Retrieval on Electronic Dance Music标题：Loopy：一种便于研究的电子舞曲音乐信息检索MIX框架链接：https://arxiv.org/abs/2305.01051作者：Xinyu Li机构：New York University, New York, USA备注：Submitted to ACM MM 2023摘要：近年来，随着深度学习的发展，音乐信息检索（MIR）经历了爆炸性的发展。然而，像电子舞曲（EDM）这样的音乐类型与其他类型相比一直相对较少研究。考虑到其广泛的应用，我们提出了一个Python包自动EDM音频生成作为基础设施的MIR EDM歌曲，以减轻获取标记的数据的难度。它是一个方便的工具，可以很容易地连接到许多符号音乐生成管道的末端。在这个包中，我们提供了一个框架来构建专业级的模板，可以从指定的旋律和和弦渲染一个制作良好的轨道，或产生大规模的轨道，只给定一个特定的关键由我们的概率符号旋律生成器。实验表明，我们的混合可以达到相同的质量的原始参考歌曲由世界著名的艺术家，就主观和客观的标准。我们的代码可以在这个仓库中访问：www.example.com 和该项目的官方网站也在线 www.example.com 网站上。摘要：Music information retrieval (MIR) has gone through an explosive development with the advancement of deep learning in recent years. However, music genres like electronic dance music (EDM) has always been relatively less investigated compared to others. Considering its wide range of applications, we present a Python package for automated EDM audio generation as an infrastructure for MIR for EDM songs, to mitigate the difficulty of acquiring labelled data. It is a convenient tool that could be easily concatenated to the end of many symbolic music generation pipelines. Inside this package, we provide a framework to build professional-level templates that could render a well-produced track from specified melody and chords, or produce massive tracks given only a specific key by our probabilistic symbolic melody generator. Experiments show that our mixes could achieve the same quality of the original reference songs produced by world-famous artists, with respect to both subjective and objective criteria. Our code is accessible in this repository: https://github.com/Gariscat/loopy and the official site of the project is also online https://loopy4edm.com .【11】 Deep Learning for Joint Acoustic Echo and Acoustic Howling Suppression in Hybrid Meetings标题：混合会议中联合声学回声和声学咆哮抑制的深度学习链接：https://arxiv.org/abs/2305.01637作者：Hao Zhang,Meng Yu,Dong Yu摘要：混合会议在后COVID时期变得越来越必要，也为解决音频相关问题带来了新的挑战。特别地，混合会议中的声学回声和声学啸叫之间的相互作用使得它们的联合抑制变得困难。本文提出了一种深度学习方法来解决这个问题，通过使用教师强制训练策略将循环反馈抑制过程制定为瞬时语音分离任务。具体地，利用自关注递归神经网络从具有可访问和学习的参考信号的麦克风记录中提取目标语音，从而同时抑制声学回声和声学啸叫。不同的输入信号和损失函数的组合进行了研究，以提高性能。实验结果表明，该方法能够有效地抑制混合会议中的回声和啸叫。摘要：Hybrid meetings have become increasingly necessary during the post-COVID period and also brought new challenges for solving audio-related problems. In particular, the interplay between acoustic echo and acoustic howling in a hybrid meeting makes the joint suppression of them difficult. This paper proposes a deep learning approach to tackle this problem by formulating a recurrent feedback suppression process as an instantaneous speech separation task using the teacher-forced training strategy. Specifically, a self-attentive recurrent neural network is utilized to extract the target speech from microphone recordings with accessible and learned reference signals, thus suppressing acoustic echo and acoustic howling simultaneously. Different combinations of input signals and loss functions have been investigated for performance improvement. Experimental results demonstrate the effectiveness of the proposed method for suppressing echo and howling jointly in hybrid meetings.【12】 Lessons Learned in ATCO2: 5000 hours of Air Traffic Control Communications for Robust Automatic Speech Recognition and Understanding标题：在ATCO2中学到的经验：5000小时的空中交通管制通信，以实现稳健的自动语音识别和理解链接：https://arxiv.org/abs/2305.01155作者：Juan Zuluaga-Gomez,Iuliia Nigmatulina,Amrutha Prasad,Petr Motlicek,Driss Khalil,Srikanth Madikeri,Allan Tart,Igor Szoke,Vincent Lenders,Mickael Rigault,Khalid Choukri机构：Idiap Research Institute, Martigny, Switzerland, Ecole Polytechnique Fédérale de Lausanne, Switzerland, University of Zurich, Switzerland, Brno University of Technology, Brno, Czechia, OpenSky Network, Burgdorf, Switzerland备注：Manuscript under review摘要：空中交通管制员（ATCos）和飞行员之间的语音通信对于确保安全和高效的空中交通管制（ATC）至关重要。此任务需要ATCos的高度意识，并且可能是繁琐且容易出错的。最近已经尝试将人工智能（AI）集成到ATC中以减少ATCos的工作量。然而，用于ATC的数据驱动AI系统的开发需要大规模的注释数据集，这是目前该领域缺乏的。本文探讨了从ATCO 2项目中吸取的经验教训，该项目旨在开发一个独特的平台，以实时收集和预处理来自空域的大量ATC数据。音频和监控数据是从一个志愿者社区拥有的VHF接收器的公共无线电频率频道收集的，后来上传到Opensky网络服务器，这可以被认为是一个“无限的数据来源”。此外，本文回顾了ATCO 2合作伙伴以前的工作，包括（i）强大的自动语音识别，（ii）自然语言处理，（iii）ATC通信的英语语言识别，以及（iv）ADS-B等监视数据的集成。我们相信，在ATCO 2项目期间开发的管道，以及其数据的开源，将鼓励ATC领域的研究。ATCO 2语料库的样本可在以下网站上获得：https：//www.atco2.org/data，而完整的语料库可以通过ELDA在www. example上购买 www.example.com 网站上。我们证明，ATCO 2是一个合适的数据集，以开发ASR引擎时，很少或接近没有ATC域内数据可用。例如，使用CNN-TDNNf kaldi模型，我们达到了低至17的性能。9%和24.9%的WER在公共ATC数据集上是6。6/7.6%优于“域外”但有监督的CNN-TDNNf模型。摘要：Voice communication between air traffic controllers (ATCos) and pilots is critical for ensuring safe and efficient air traffic control (ATC). This task requires high levels of awareness from ATCos and can be tedious and error-prone. Recent attempts have been made to integrate artificial intelligence (AI) into ATC in order to reduce the workload of ATCos. However, the development of data-driven AI systems for ATC demands large-scale annotated datasets, which are currently lacking in the field. This paper explores the lessons learned from the ATCO2 project, a project that aimed to develop a unique platform to collect and preprocess large amounts of ATC data from airspace in real time. Audio and surveillance data were collected from publicly accessible radio frequency channels with VHF receivers owned by a community of volunteers and later uploaded to Opensky Network servers, which can be considered an "unlimited source" of data. In addition, this paper reviews previous work from ATCO2 partners, including (i) robust automatic speech recognition, (ii) natural language processing, (iii) English language identification of ATC communications, and (iv) the integration of surveillance data such as ADS-B. We believe that the pipeline developed during the ATCO2 project, along with the open-sourcing of its data, will encourage research in the ATC field. A sample of the ATCO2 corpus is available on the following website: https://www.atco2.org/data, while the full corpus can be purchased through ELDA at http://catalog.elra.info/en-us/repository/browse/ELRA-S0484. We demonstrated that ATCO2 is an appropriate dataset to develop ASR engines when little or near to no ATC in-domain data is available. For instance, with the CNN-TDNNf kaldi model, we reached the performance of as low as 17.9% and 24.9% WER on public ATC datasets which is 6.6/7.6% better than "out-of-domain" but supervised CNN-TDNNf model.eess.AS音频处理【1】 Deep Learning for Joint Acoustic Echo and Acoustic Howling Suppression in Hybrid Meetings标题：混合会议中联合声学回声和声学咆哮抑制的深度学习链接：https://arxiv.org/abs/2305.01637*与cs.SD语音【11】为同一篇作者：Hao Zhang,Meng Yu,Dong Yu摘要：混合会议在后COVID时期变得越来越必要，也为解决音频相关问题带来了新的挑战。特别地，混合会议中的声学回声和声学啸叫之间的相互作用使得它们的联合抑制变得困难。本文提出了一种深度学习方法来解决这个问题，通过使用教师强制训练策略将循环反馈抑制过程制定为瞬时语音分离任务。具体地，利用自关注递归神经网络从具有可访问和学习的参考信号的麦克风记录中提取目标语音，从而同时抑制声学回声和声学啸叫。不同的输入信号和损失函数的组合进行了研究，以提高性能。实验结果表明，该方法能够有效地抑制混合会议中的回声和啸叫。摘要：Hybrid meetings have become increasingly necessary during the post-COVID period and also brought new challenges for solving audio-related problems. In particular, the interplay between acoustic echo and acoustic howling in a hybrid meeting makes the joint suppression of them difficult. This paper proposes a deep learning approach to tackle this problem by formulating a recurrent feedback suppression process as an instantaneous speech separation task using the teacher-forced training strategy. Specifically, a self-attentive recurrent neural network is utilized to extract the target speech from microphone recordings with accessible and learned reference signals, thus suppressing acoustic echo and acoustic howling simultaneously. Different combinations of input signals and loss functions have been investigated for performance improvement. Experimental results demonstrate the effectiveness of the proposed method for suppressing echo and howling jointly in hybrid meetings.【2】 Lessons Learned in ATCO2: 5000 hours of Air Traffic Control Communications for Robust Automatic Speech Recognition and Understanding标题：在ATCO2中学到的经验：5000小时的空中交通管制通信，以实现稳健的自动语音识别和理解链接：https://arxiv.org/abs/2305.01155*与cs.SD语音【12】为同一篇作者：Juan Zuluaga-Gomez,Iuliia Nigmatulina,Amrutha Prasad,Petr Motlicek,Driss Khalil,Srikanth Madikeri,Allan Tart,Igor Szoke,Vincent Lenders,Mickael Rigault,Khalid Choukri机构：Idiap Research Institute, Martigny, Switzerland, Ecole Polytechnique Fédérale de Lausanne, Switzerland, University of Zurich, Switzerland, Brno University of Technology, Brno, Czechia, OpenSky Network, Burgdorf, Switzerland备注：Manuscript under review摘要：空中交通管制员（ATCos）和飞行员之间的语音通信对于确保安全和高效的空中交通管制（ATC）至关重要。此任务需要ATCos的高度意识，并且可能是繁琐且容易出错的。最近已经尝试将人工智能（AI）集成到ATC中以减少ATCos的工作量。然而，用于ATC的数据驱动AI系统的开发需要大规模的注释数据集，这是目前该领域缺乏的。本文探讨了从ATCO 2项目中吸取的经验教训，该项目旨在开发一个独特的平台，以实时收集和预处理来自空域的大量ATC数据。音频和监控数据是从一个志愿者社区拥有的VHF接收器的公共无线电频率频道收集的，后来上传到Opensky网络服务器，这可以被认为是一个“无限的数据来源”。此外，本文回顾了ATCO 2合作伙伴以前的工作，包括（i）强大的自动语音识别，（ii）自然语言处理，（iii）ATC通信的英语语言识别，以及（iv）ADS-B等监视数据的集成。我们相信，在ATCO 2项目期间开发的管道，以及其数据的开源，将鼓励ATC领域的研究。ATCO 2语料库的样本可在以下网站上获得：https：//www.atco2.org/data，而完整的语料库可以通过ELDA在www. example上购买 www.example.com 网站上。我们证明，ATCO 2是一个合适的数据集，以开发ASR引擎时，很少或接近没有ATC域内数据可用。例如，使用CNN-TDNNf kaldi模型，我们达到了低至17的性能。9%和24.9%的WER在公共ATC数据集上是6。6/7.6%优于“域外”但有监督的CNN-TDNNf模型。摘要：Voice communication between air traffic controllers (ATCos) and pilots is critical for ensuring safe and efficient air traffic control (ATC). This task requires high levels of awareness from ATCos and can be tedious and error-prone. Recent attempts have been made to integrate artificial intelligence (AI) into ATC in order to reduce the workload of ATCos. However, the development of data-driven AI systems for ATC demands large-scale annotated datasets, which are currently lacking in the field. This paper explores the lessons learned from the ATCO2 project, a project that aimed to develop a unique platform to collect and preprocess large amounts of ATC data from airspace in real time. Audio and surveillance data were collected from publicly accessible radio frequency channels with VHF receivers owned by a community of volunteers and later uploaded to Opensky Network servers, which can be considered an "unlimited source" of data. In addition, this paper reviews previous work from ATCO2 partners, including (i) robust automatic speech recognition, (ii) natural language processing, (iii) English language identification of ATC communications, and (iv) the integration of surveillance data such as ADS-B. We believe that the pipeline developed during the ATCO2 project, along with the open-sourcing of its data, will encourage research in the ATC field. A sample of the ATCO2 corpus is available on the following website: https://www.atco2.org/data, while the full corpus can be purchased through ELDA at http://catalog.elra.info/en-us/repository/browse/ELRA-S0484. We demonstrated that ATCO2 is an appropriate dataset to develop ASR engines when little or near to no ATC in-domain data is available. For instance, with the CNN-TDNNf kaldi model, we reached the performance of as low as 17.9% and 24.9% WER on public ATC datasets which is 6.6/7.6% better than "out-of-domain" but supervised CNN-TDNNf model.【3】 Basic syntax from speech: Spontaneous concatenation in unsupervised deep neural networks标题：来自语音的基本句法：无监督深度神经网络中的自发连接链接：https://arxiv.org/abs/2305.01626*与cs.SD语音【1】为同一篇作者：Gašper Beguš,Thomas Lu,Zili Wang机构：UC Berkeley摘要：句法的计算模型主要是基于文本的。在这里，我们建议，基本的语法可以直接从原始语音建模在一个完全无监督的方式。我们专注于语法中最普遍和最基本的属性之一--连接。我们引入自发级联：这是一种现象，在单个单词的声学记录上训练的卷积神经网络（CNN）开始生成具有两个甚至三个单词的输出，而无需访问输入中具有多个单词的数据。此外，在两个单词上训练的网络学习将单词嵌入到新的未观察到的单词组合中。据我们所知，这是在生成对抗网络设置中对原始语音进行训练的CNN的一个以前未报告的属性，并且对于我们理解这些架构如何学习以及建模语法及其从原始声学输入的演变都有影响。摘要：Computational models of syntax are predominantly text-based. Here we propose that basic syntax can be modeled directly from raw speech in a fully unsupervised way. We focus on one of the most ubiquitous and basic properties of syntax -- concatenation. We introduce spontaneous concatenation: a phenomenon where convolutional neural networks (CNNs) trained on acoustic recordings of individual words start generating outputs with two or even three words concatenated without ever accessing data with multiple words in the input. Additionally, networks trained on two words learn to embed words into novel unobserved word combinations. To our knowledge, this is a previously unreported property of CNNs trained on raw speech in the Generative Adversarial Network setting and has implications both for our understanding of how these architectures learn as well as for modeling syntax and its evolution from raw acoustic inputs.【4】 A Study on the Integration of Pipeline and E2E SLU systems for Spoken Semantic Parsing toward STOP Quality Challenge标题：面向Stop Quality挑战的口语语义分析流水线与E2E SLU系统集成研究链接：https://arxiv.org/abs/2305.01620*与cs.SD语音【2】为同一篇作者：Siddhant Arora,Hayato Futami,Shih-Lun Wu,Jessica Huynh,Yifan Peng,Yosuke Kashiwagi,Emiru Tsunoo,Brian Yan,Shinji Watanabe机构：Carnegie Mellon University,Sony Group Corporation, Japan备注：First Place in Track 1 of STOP Challenge, which is part of ICASSP Signal Processing Grand Challenge 2023摘要：最近，人们一直在努力为口语理解（SLU）引入新的基准任务，如语义解析。在本文中，我们描述了我们提出的口语语义解析系统的质量轨道（轨道1）在口语理解大挑战，这是ICASSP信号处理大挑战2023的一部分。我们尝试使用端到端和管道系统来执行此任务。强大的自动语音识别（ASR）模型（如Whisper）和预训练的语言模型（LM）（如BART）在我们的SLU框架中使用，以提高性能。我们还研究了各种模型的输出水平组合，以获得80的精确匹配精度。8，在挑战赛中获得第一名。摘要：Recently there have been efforts to introduce new benchmark tasks for spoken language understanding (SLU), like semantic parsing. In this paper, we describe our proposed spoken semantic parsing system for the quality track (Track 1) in Spoken Language Understanding Grand Challenge which is part of ICASSP Signal Processing Grand Challenge 2023. We experiment with both end-to-end and pipeline systems for this task. Strong automatic speech recognition (ASR) models like Whisper and pretrained Language models (LM) like BART are utilized inside our SLU framework to boost performance. We also investigate the output level combination of various models to get an exact match accuracy of 80.8, which won the 1st place at the challenge.【5】 Self-supervised learning for infant cry analysis标题：用于婴儿哭闹分析的自我监督学习链接：https://arxiv.org/abs/2305.01578*与cs.SD语音【3】为同一篇作者：Arsenii Gorin,Cem Subakan,Sajjad Abdoli,Junhao Wang,Samantha Latremouille,Charles Onu机构：⋆Ubenwa Health, Montr´eal, Canada, ♮Universit´e Laval, Qu´ebec City, Canada, ♯Concordia University, Montr´eal, Canada, ♭Mila-Qu´ebec AI Institute, Montr´eal, Canada备注：Accepted to IEEE ICASSP 2023 workshop Self-supervision in Audio, Speech and Beyond摘要：在本文中，我们探讨了自我监督学习（SSL）分析的第一个同类数据库的哭记录包含临床指征的1000多个新生儿。具体来说，我们的目标是基于哭泣的神经损伤检测以及识别哭泣触发器，如疼痛，饥饿和不适。在医疗环境中注释大型数据库是昂贵且耗时的，通常需要几位专家多年的合作。利用大量未标记的音频数据来学习有用的表示可以降低构建鲁棒模型的成本，并最终降低临床解决方案的成本。在这项工作中，我们在大型音频数据集上进行了卷积神经网络的自监督预训练实验。我们发现，SSL对比损失（SimCLR）的预训练在神经损伤和哭泣触发方面的表现明显优于监督预训练。此外，我们展示了进一步的性能增益，通过SSL为基础的域适应使用未标记的婴儿哭声。我们还表明，使用这种基于SSL的预训练适应哭泣的声音减少了对整个系统的标记数据的需要。摘要：In this paper, we explore self-supervised learning (SSL) for analyzing a first-of-its-kind database of cry recordings containing clinical indications of more than a thousand newborns. Specifically, we target cry-based detection of neurological injury as well as identification of cry triggers such as pain, hunger, and discomfort. Annotating a large database in the medical setting is expensive and time-consuming, typically requiring the collaboration of several experts over years. Leveraging large amounts of unlabeled audio data to learn useful representations can lower the cost of building robust models and, ultimately, clinical solutions. In this work, we experiment with self-supervised pre-training of a convolutional neural network on large audio datasets. We show that pre-training with SSL contrastive loss (SimCLR) performs significantly better than supervised pre-training for both neuro injury and cry triggers. In addition, we demonstrate further performance gains through SSL-based domain adaptation using unlabeled infant cries. We also show that using such SSL-based pre-training for adaptation to cry sounds decreases the need for labeled data of the overall system.【6】 Multitask learning in Audio Captioning: a sentence embedding regression loss acts as a regularizer标题：音频字幕中的多任务学习：嵌入回归损失的句子充当正则化链接：https://arxiv.org/abs/2305.01482*与cs.SD语音【4】为同一篇作者：Etienne Labbé,Julien Pinquier,Thomas Pellegrini机构：IRIT, Universit´e de Toulouse, CNRS, UT, Toulouse, France摘要：在这项工作中，我们建议研究使用句子嵌入回归损失组件训练的模型在自动音频字幕任务中的性能。该任务旨在构建可以用自然语言编写的单个句子描述音频内容的系统。大多数系统都是用标准的交叉熵损失来训练的，它没有考虑句子的语义接近度。我们发现，添加一个句子嵌入损失项减少了过拟合，但也增加了SPIDER从0。397到0.418在我们的第一个设置上的AudioCaps语料库。当我们增加权重衰减值时，我们发现我们的模型更接近当前最先进的方法，SPIDEr得分高达0。444与0相比。475分。此外，该模型使用少八倍的可训练参数。在这种训练设置中，句子嵌入损失对模型性能没有更多的影响。摘要：In this work, we propose to study the performance of a model trained with a sentence embedding regression loss component for the Automated Audio Captioning task. This task aims to build systems that can describe audio content with a single sentence written in natural language. Most systems are trained with the standard Cross-Entropy loss, which does not take into account the semantic closeness of the sentence. We found that adding a sentence embedding loss term reduces overfitting, but also increased SPIDEr from 0.397 to 0.418 in our first setting on the AudioCaps corpus. When we increased the weight decay value, we found our model to be much closer to the current state- of-the-art methods, with a SPIDEr score up to 0.444 compared to a 0.475 score. Moreover, this model uses eight times less trainable parameters. In this training setting, the sentence embedding loss has no more impact on the model performance.【7】 Deep Learning Based Multimodal with Two-phase Training Strategy for Daily Life Video Classification标题：基于深度学习的多模式两阶段训练日常生活视频分类策略链接：https://arxiv.org/abs/2305.01476*与cs.SD语音【5】为同一篇作者：Lam Pham,Trang Le,Cam Le,Dat Ngo,Weissenfeld Axel,Alexander Schindler机构：Le is with HCM University of Technology, Ngo is with School of Computer Science and Electronic Engineering, University of Essex摘要：在本文中，我们提出了一个基于深度学习的多模态系统，用于分类日常生活视频。为了训练系统，我们提出了一个两阶段的训练策略。在第一个训练阶段（第一阶段），我们从原始视频中提取音频和视觉（图像）数据。然后，我们使用独立的基于深度学习的模型来训练音频数据和视觉数据。在训练过程之后，我们通过从预训练的深度学习模型中提取特征图来获得音频嵌入和视觉嵌入。在第二个训练阶段（第二阶段），我们训练一个融合层来组合音频/视频嵌入，并训练一个密集层来将组合的嵌入分类到目标日常场景中。我们在DCASE（IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events）2021 Task 1B Development的基准数据集上进行了广泛的实验，达到了80的最佳分类准确率。5%、91. 8%和95。3%仅具有音频数据，仅具有视觉数据，分别具有音频和视觉数据。最高分类准确率达95.3%，改善了17。9%，与DCASE基线相比，显示出非常具有竞争力的国家的最先进的系统。摘要：In this paper, we present a deep learning based multimodal system for classifying daily life videos. To train the system, we propose a two-phase training strategy. In the first training phase (Phase I), we extract the audio and visual (image) data from the original video. We then train the audio data and the visual data with independent deep learning based models. After the training processes, we obtain audio embeddings and visual embeddings by extracting feature maps from the pre-trained deep learning models. In the second training phase (Phase II), we train a fusion layer to combine the audio/visual embeddings and a dense layer to classify the combined embedding into target daily scenes. Our extensive experiments, which were conducted on the benchmark dataset of DCASE (IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events) 2021 Task 1B Development, achieved the best classification accuracy of 80.5%, 91.8%, and 95.3% with only audio data, with only visual data, both audio and visual data, respectively. The highest classification accuracy of 95.3% presents an improvement of 17.9% compared with DCASE baseline and shows very competitive to the state-of-the-art systems.【8】 Long-Term Rhythmic Video Soundtracker标题：长期有节奏的视频音响追踪器链接：https://arxiv.org/abs/2305.01319*与cs.SD语音【6】为同一篇作者：Jiashuo Yu,Yaohui Wang,Xinyuan Chen,Xiao Sun,Yu Qiao备注：ICML2023摘要：我们考虑的问题，产生的音乐音轨同步有节奏的视觉线索。大多数现有的作品依赖于预先定义的音乐表示，导致生成的灵活性和复杂性的无能。直接生成视频调节波形的其他方法遭受有限的场景、短长度和不稳定的生成质量。为此，我们提出了长期的节奏视频声音跟踪器（LORIS），一个新的框架来合成长期的条件波形。具体而言，我们的框架包括一个潜在的条件扩散概率模型来执行波形合成。此外，提出了一系列的上下文感知的条件编码器，考虑到时间信息的长期生成。值得注意的是，我们扩展我们的模型的适用性从舞蹈到多个运动场景，如自由体操和花样滑冰。为了进行全面的评估，我们建立了一个基准的节奏视频音轨，包括预处理的数据集，改进的评估指标，和强大的生成基线。大量的实验表明，我们的模型生成长期的音轨与国家的最先进的音乐质量和节奏的对应关系。代码可在\url{https：//github.com/OpenGVLab/LORIS}。摘要：We consider the problem of generating musical soundtracks in sync with rhythmic visual cues. Most existing works rely on pre-defined music representations, leading to the incompetence of generative flexibility and complexity. Other methods directly generating video-conditioned waveforms suffer from limited scenarios, short lengths, and unstable generation quality. To this end, we present Long-Term Rhythmic Video Soundtracker (LORIS), a novel framework to synthesize long-term conditional waveforms. Specifically, our framework consists of a latent conditional diffusion probabilistic model to perform waveform synthesis. Furthermore, a series of context-aware conditioning encoders are proposed to take temporal information into consideration for a long-term generation. Notably, we extend our model's applicability from dances to multiple sports scenarios such as floor exercise and figure skating. To perform comprehensive evaluations, we establish a benchmark for rhythmic video soundtracks including the pre-processed dataset, improved evaluation metrics, and robust generative baselines. Extensive experiments show that our model generates long-term soundtracks with state-of-the-art musical quality and rhythmic correspondence. Codes are available at \url{https://github.com/OpenGVLab/LORIS}.【9】 AQ-GT: a Temporally Aligned and Quantized GRU-Transformer for Co-Speech Gesture Synthesis标题：AQ-GT：一种用于协同语音手势合成的时间对准量化GRU转换器链接：https://arxiv.org/abs/2305.01241*与cs.SD语音【7】为同一篇作者：Hendric Voß,Stefan Kopp机构：Social Cognitive Systems Group, Bielefeld University, Germany摘要：现实的和上下文相关的协同语音手势的生成是一个具有挑战性的，但越来越重要的任务，在创建多模态人工代理。先前的方法集中于学习共同语音手势表示和所产生的运动之间的直接对应关系，这在人类评估期间创建了看似自然但通常不令人信服的手势。我们提出了一种方法来预训练部分手势序列使用生成对抗网络与量化管道。由此产生的码本向量作为输入和输出，在我们的框架，形成的基础上生成和重建的手势。通过学习潜在空间表示的映射而不是直接将其映射到矢量表示，该框架有助于生成高度逼真和富有表现力的手势，这些手势紧密地复制人类运动和行为，同时避免生成过程中的伪影。我们评估我们的方法，通过比较它与已建立的方法生成语音手势，以及与现有的人类行为的数据集。我们还进行了消融研究，以评估我们的发现。结果表明，我们的方法优于目前的最先进的一个明确的保证金，是部分无法区分人类手势。我们公开了我们的数据管道和生成框架。摘要：The generation of realistic and contextually relevant co-speech gestures is a challenging yet increasingly important task in the creation of multimodal artificial agents. Prior methods focused on learning a direct correspondence between co-speech gesture representations and produced motions, which created seemingly natural but often unconvincing gestures during human assessment. We present an approach to pre-train partial gesture sequences using a generative adversarial network with a quantization pipeline. The resulting codebook vectors serve as both input and output in our framework, forming the basis for the generation and reconstruction of gestures. By learning the mapping of a latent space representation as opposed to directly mapping it to a vector representation, this framework facilitates the generation of highly realistic and expressive gestures that closely replicate human movement and behavior, while simultaneously avoiding artifacts in the generation process. We evaluate our approach by comparing it with established methods for generating co-speech gestures as well as with existing datasets of human behavior. We also perform an ablation study to assess our findings. The results show that our approach outperforms the current state of the art by a clear margin and is partially indistinguishable from human gesturing. We make our data pipeline and the generation framework publicly available.【10】 The Pipeline System of ASR and NLU with MLM-based Data Augmentation toward STOP Low-resource Challenge标题：基于MLM数据增强的ASR和NLU流水线系统应对低资源挑战链接：https://arxiv.org/abs/2305.01194*与cs.SD语音【8】为同一篇作者：Hayato Futami,Jessica Huynh,Siddhant Arora,Shih-Lun Wu,Yosuke Kashiwagi,Yifan Peng,Brian Yan,Emiru Tsunoo,Shinji Watanabe机构：Sony Group Corporation, Japan ,Carnegie Mellon University, USA备注：To be appeared at ICASSP2023摘要：None摘要：This paper describes our system for the low-resource domain adaptation track (Track 3) in Spoken Language Understanding Grand Challenge, which is a part of ICASSP Signal Processing Grand Challenge 2023. In the track, we adopt a pipeline approach of ASR and NLU. For ASR, we fine-tune Whisper for each domain with upsampling. For NLU, we fine-tune BART on all the Track3 data and then on low-resource domain data. We apply masked LM (MLM) -based data augmentation, where some of input tokens and corresponding target labels are replaced using MLM. We also apply a retrieval-based approach, where model input is augmented with similar training samples. As a result, we achieved exact match (EM) accuracy 63.3/75.0 (average: 69.15) for reminder/weather domain, and won the 1st place at the challenge.【11】 Contrastive Speech Mixup for Low-resource Keyword Spotting标题：用于低资源关键词检测的对比语音混合链接：https://arxiv.org/abs/2305.01170*与cs.SD语音【9】为同一篇作者：Dianwen Ng,Ruixi Zhang,Jia Qi Yip,Chong Zhang,Yukun Ma,Trung Hieu Nguyen,Chongjia Ni,Eng Siong Chng,Bin Ma机构：Alibaba Group, School of Computer Science and Engineering, Nanyang Technological University, Singapore备注：Accepted by ICASSP 2023摘要：智能设备中大多数现有的基于神经的关键词识别模型（KWS）需要数千个训练样本来学习一个像样的音频表示。然而，随着智能设备变得更加个性化的需求不断增长，KWS模型需要快速适应较小的用户样本。为了应对这一挑战，我们提出了一个对比语音混合（CosMix）学习算法的低资源KWS。CosMix引入了一个辅助对比损失现有的混合增强技术，以最大限度地提高原始预混合样本和增强样本之间的相对相似性。我们的目标是注入增强约束，以引导模型从两个增强视图（即：即噪声混合和干净的预混合话语）。我们在Google Speech Command数据集上进行实验，将训练集的大小调整为2。每个关键字5分钟，模拟低资源条件。我们的实验结果表明，在多个模型的性能，表现出我们的方法的有效性一致的改善。摘要：Most of the existing neural-based models for keyword spotting (KWS) in smart devices require thousands of training samples to learn a decent audio representation. However, with the rising demand for smart devices to become more personalized, KWS models need to adapt quickly to smaller user samples. To tackle this challenge, we propose a contrastive speech mixup (CosMix) learning algorithm for low-resource KWS. CosMix introduces an auxiliary contrastive loss to the existing mixup augmentation technique to maximize the relative similarity between the original pre-mixed samples and the augmented samples. The goal is to inject enhancing constraints to guide the model towards simpler but richer content-based speech representations from two augmented views (i.e. noisy mixed and clean pre-mixed utterances). We conduct our experiments on the Google Speech Command dataset, where we trim the size of the training set to as small as 2.5 mins per keyword to simulate a low-resource condition. Our experimental results show a consistent improvement in the performance of multiple models, which exhibits the effectiveness of our method.【12】 LooPy: A Research-Friendly Mix Framework for Music Information Retrieval on Electronic Dance Music标题：Loopy：一种便于研究的电子舞曲音乐信息检索MIX框架链接：https://arxiv.org/abs/2305.01051*与cs.SD语音【10】为同一篇作者：Xinyu Li机构：New York University, New York, USA备注：Submitted to ACM MM 2023摘要：近年来，随着深度学习的发展，音乐信息检索（MIR）经历了爆炸性的发展。然而，像电子舞曲（EDM）这样的音乐类型与其他类型相比一直相对较少研究。考虑到其广泛的应用，我们提出了一个Python包自动EDM音频生成作为基础设施的MIR EDM歌曲，以减轻获取标记的数据的难度。它是一个方便的工具，可以很容易地连接到许多符号音乐生成管道的末端。在这个包中，我们提供了一个框架来构建专业级的模板，可以从指定的旋律和和弦渲染一个制作良好的轨道，或产生大规模的轨道，只给定一个特定的关键由我们的概率符号旋律生成器。实验表明，我们的混合可以达到相同的质量的原始参考歌曲由世界著名的艺术家，就主观和客观的标准。我们的代码可以在这个仓库中访问：www.example.com 和该项目的官方网站也在线 www.example.com 网站上。摘要：Music information retrieval (MIR) has gone through an explosive development with the advancement of deep learning in recent years. However, music genres like electronic dance music (EDM) has always been relatively less investigated compared to others. Considering its wide range of applications, we present a Python package for automated EDM audio generation as an infrastructure for MIR for EDM songs, to mitigate the difficulty of acquiring labelled data. It is a convenient tool that could be easily concatenated to the end of many symbolic music generation pipelines. Inside this package, we provide a framework to build professional-level templates that could render a well-produced track from specified melody and chords, or produce massive tracks given only a specific key by our probabilistic symbolic melody generator. Experiments show that our mixes could achieve the same quality of the original reference songs produced by world-famous artists, with respect to both subjective and objective criteria. Our code is accessible in this repository: https://github.com/Gariscat/loopy and the official site of the project is also online https://loopy4edm.com .
每日语音速递
