今天跟大家分享一篇语音相关的论文合集：cs.SD语音9篇，eess.AS音频处理10篇。
【1】 Learning Hierarchical Metrical Structure Beyond Measures
标题：学习超越度量的层次化度量结构
链接：https://arxiv.org/abs/2209.10259
作者：Junyan Jiang,Daniel Chin,Yixiao Zhang,Gus Xia机构：Music X Lab, NYU Shanghai, MBZUAI, Centre for Digital Music, QMUL备注：Accepted at the International Society for Music Information Retrieval (ISMIR), 2022摘要：音乐包含着超越节拍和小节的层次结构。层次结构的音乐标注在音乐信息检索和计算机音乐学中具有重要的应用价值，但在现有的数字音乐数据库中却很少见到。本文提出了一种基于数据驱动的自动抽取韵律层次结构的方法。提出了一种基于时间卷积网络-条件随机场（TCN-CRF）结构的新模型。给定一个符号乐谱，我们的模型以节拍量化的形式接收任意数量的声部，并预测出从下拍级到小节级的4级分层韵律结构。我们还使用RWC-POP MIDI文件注释数据集，以便于训练和评估。实验结果表明，在不同的编排设置下，该方法的性能优于基于规则的方法。我们还对模型预测进行了一些简单的音乐学分析。所有的演示、数据集和预先训练的模型都可以在Github上公开获得。摘要：Music contains hierarchical structures beyond beats and measures. While hierarchical structure annotations are helpful for music information retrieval and computer musicology, such annotations are scarce in current digital music databases. In this paper, we explore a data-driven approach to automatically extract hierarchical metrical structures from scores. We propose a new model with a Temporal Convolutional Network-Conditional Random Field (TCN-CRF) architecture. Given a symbolic music score, our model takes in an arbitrary number of voices in a beat-quantized form, and predicts a 4-level hierarchical metrical structure from downbeat-level to section-level. We also annotate a dataset using RWC-POP MIDI files to facilitate training and evaluation. We show by experiments that the proposed method performs better than the rule-based approach under different orchestration settings. We also perform some simple musicological analysis on the model predictions. All demos, datasets and pre-trained models are publicly available on Github.
【2】 Dynamic Time-Alignment of Dimensional Annotations of Emotion using Recurrent Neural Networks
标题：基于递归神经网络的情感维度标注的动态时间对齐
链接：https://arxiv.org/abs/2209.10223
作者：Sina Alisamir,Fabien Ringeval,Francois Portet机构：Atos & LIG, Grenoble, France, Grenoble INP, LIG, Univ. Grenoble Alpes, Inria, CNRS, François Portet摘要：大多数自动情感识别系统利用情感的时间连续注释来提供如在现实生活交互中观察到的自发表情的细粒度描述。由于情绪是相当主观的，它的注释通常由几个注释者执行，他们提供给定维度的轨迹，即描述诸如唤醒或效价等维度的时间连续序列。然而，同一表情的注释在注释器之间很少在时间或值上一致，这在用于学习情感的预测模型的轨迹中增加了偏差和延迟。因此，我们提出了一种方法，可以动态补偿注释之间的不一致性，并使用递归神经网络将轨迹与相应的声学特征同步。实验评估是在几个情感数据集上进行的，这些数据集包括中国、法国、德国和匈牙利的参与者，他们在无噪声条件下或在野外进行远程交互。实验结果表明，该方法可以显著提高标注者之间的一致性，以及轨迹和音频特征之间的相关性，无论是唤醒还是效价。此外，使用简单的轻量模型在这些维度的自动预测中获得了改进，特别是对于无噪声条件下的效价和在野外捕获的记录的唤醒。摘要：Most automatic emotion recognition systems exploit time-continuous annotations of emotion to provide fine-grained descriptions of spontaneous expressions as observed in real-life interactions. As emotion is rather subjective, its annotation is usually performed by several annotators who provide a trace for a given dimension, i.e. a time-continuous series describing a dimension such as arousal or valence. However, annotations of the same expression are rarely consistent between annotators, either in time or in value, which adds bias and delay in the trace that is used to learn predictive models of emotion. We therefore propose a method that can dynamically compensate inconsistencies across annotations and synchronise the traces with the corresponding acoustic features using Recurrent Neural Networks. Experimental evaluations were carried on several emotion data sets that include Chinese, French, German, and Hungarian participants who interacted remotely in either noise-free conditions or in-the-wild. The results show that our method can significantly increase inter-annotator agreement, as well as correlation between traces and audio features, for both arousal and valence. In addition, improvements are obtained in the automatic prediction of these dimensions using simple light-weight models, especially for valence in noise-free conditions, and arousal for recordings captured in-the-wild.
【3】 Setting the rhythm scene: deep learning-based drum loop generation from arbitrary language cues
标题：设置节奏场景：基于深度学习的任意语言提示的鼓循环生成
链接：https://arxiv.org/abs/2209.10016
机构：Independent Researcher摘要：生成式人工智能模型可以成为音乐创作和现场表演的宝贵辅助工具，既可以帮助专业音乐家，也可以帮助业余爱好者实现音乐创作过程的民主化。在此，我们提出一种新颖的方法，给定一个英语单词或短语，生成体现给定语言线索的“情绪”的4件鼓图案的2个罗盘，或者可以用于由语言线索描述的视听场景。我们将此工具设想为电子音乐和视听配乐制作的作曲辅助工具，或现场表演的即兴创作工具。为了产生该模型的训练样本，除了人工标注“场景”或“情绪”术语之外，我们还设计了一种新颖的方法来提取任何歌曲的共识鼓点。这包括一个2小节，4件鼓模式，代表了一首歌曲的主要打击乐主题，可以导入到任何音乐循环设备或现场循环软件。这两个关键组件（从可归纳的输入生成鼓模式和一致打击乐提取）为计算机辅助作曲提供了一种新的方法，并为更全面的节奏生成提供了垫脚石。摘要：Generative artificial intelligence models can be a valuable aid to music composition and live performance, both to aid the professional musician and to help democratize the music creation process for hobbyists. Here we present a novel method that, given an English word or phrase, generates 2 compasses of a 4-piece drum pattern that embodies the "mood" of the given language cue, or that could be used for an audiovisual scene described by the language cue. We envision this tool as composition aid for electronic music and audiovisual soundtrack production, or an improvisation tool for live performance. In order to produce the training samples for this model, besides manual annotation of the "scene" or "mood" terms, we have designed a novel method to extract the consensus drum track of any song. This consists of a 2-bar, 4-piece drum pattern that represents the main percussive motif of a song, which could be imported into any music loop device or live looping software. These two key components (drum pattern generation from a generalizable input, and consensus percussion extraction) present a novel approach to computer-aided composition and provide a stepping stone for more comprehensive rhythm generation.
【4】 Meta-Learning for Adaptive Filters with Higher-Order Frequency Dependencies
标题：高阶频率相关自适应滤波器的元学习
链接：https://arxiv.org/abs/2209.09955
作者：Junkai Wu,Jonah Casebeer,Nicholas J. Bryan,Paris Smaragdis机构：♯ University of Illinois at Urbana-Champaign, ♭ Adobe Research备注：Source code and audio examples: this https URL摘要：自适应滤波器适用于许多信号处理任务，包括声学回声消除、波束形成等。自适应滤波器通常使用诸如最小均方（LMS）、递归最小二乘（RLS）或卡尔曼滤波器更新之类的算法来控制。为了简单起见，这种模型通常应用在频域中，假设频率独立处理，并且不利用高阶频率依赖性。然而，最近关于元自适应滤波器的工作已经表明，我们可以使用神经网络来控制滤波器自适应，而无需人工推导，这激发了利用这种信息的新工作。在本文中，我们提出高阶次自适应滤波器，这是对次自适应滤波器的一个关键改进，它结合了高阶频率相关性。我们演示了我们的声学回声消除方法，并开发了一系列滤波器，与竞争对手的基线相比，这些滤波器可产生数dB的改进，并且复杂性至少降低了一个数量级。此外，我们显示了我们的改进在有或没有下游语音增强器的情况下都成立。摘要：Adaptive filters are applicable to many signal processing tasks including acoustic echo cancellation, beamforming, and more. Adaptive filters are typically controlled using algorithms such as least-mean squares(LMS), recursive least squares(RLS), or Kalman filter updates. Such models are often applied in the frequency domain, assume frequency independent processing, and do not exploit higher-order frequency dependencies, for simplicity. Recent work on meta-adaptive filters, however, has shown that we can control filter adaptation using neural networks without manual derivation, motivating new work to exploit such information. In this work, we present higher-order meta-adaptive filters, a key improvement to meta-adaptive filters that incorporates higher-order frequency dependencies. We demonstrate our approach on acoustic echo cancellation and develop a family of filters that yield multi-dB improvements over competitive baselines, and are at least an order-of-magnitude less complex. Moreover, we show our improvements hold with or without a downstream speech enhancer.
【5】 An Initial study on Birdsong Re-synthesis Using Neural Vocoders
标题：用神经声码器进行BirdSong再合成的初步研究
链接：https://arxiv.org/abs/2209.10479
作者：Rhythm Bhatia,Tomi H. Kinnunen机构：University of Eastern Finland备注：To appear in 24th International Conference on Speech and Computer (SPECOM), GURUGRAM, INDIA摘要：现代语音合成使用神经声码器来直接对原始波形样本进行建模。这种增加的多功能性已经将声码器的范围从语音扩展到诸如音乐的其它领域。我们解决另一个有趣的领域的生物声学。我们提供了使用传统（WORLD）和两种神经（WaveNet自动编码器，并行WaveGAN）声码器的鸟鸣声的初步比较分析-再合成实验。我们的主观结果表明三种声码器在物种辨别方面没有差异（ABX测试）。尽管如此，WORLD声码器样本在保持鸟样品质方面被评定为较高（MOS测试）。所有的声码器都面临着音高和发声的问题。我们的结果表明了在处理低质量野生动物音频数据时的一些挑战。摘要：Modern speech synthesis uses neural vocoders to model raw waveform samples directly. This increased versatility has expanded the scope of vocoders from speech to other domains, such as music. We address another interesting domain of bio-acoustics. We provide initial comparative analysis-resynthesis experiments of birdsong using traditional (WORLD) and two neural (WaveNet autoencoder, parallel WaveGAN) vocoders. Our subjective results indicate no difference in the three vocoders in terms of species discrimination (ABX test). Nonetheless, the WORLD vocoder samples were rated higher in terms of retaining bird-like qualities (MOS test). All vocoders faced issues with pitch and voicing. Our results indicate some of the challenges in processing low-quality wildlife audio data.
【6】 Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN
标题：基于扩散概率瓦瑟斯坦去噪的普通话歌声合成
链接：https://arxiv.org/abs/2209.10446
作者：Yin-Ping Cho,Yu Tsao,Hsin-Min Wang,Yi-Wen Liu机构：National Tsing Hua University, Hsinchu, Taiwan, Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan, Institute of Information Science, Academia Sinica, Taipei, Taiwan摘要：歌唱声音合成（SVS）是从给定的乐谱中用计算机产生类似人类的歌唱声音。为了有效地实现端到端的SVS，本文采用了为高质量语音和歌唱声合成而建立的声学模型-神经元声码器结构。具体而言，本工作旨在通过将扩散去噪概率模型（DDPM）与\emph{Wasserstein}生成对抗网络（WGAN）相结合来构建声学模型的主干，从而在合成语音中追求更高级别的表现力。在所提出的声学模型之上，采用了具有集成微调的HiFi-GAN神经声码器，以确保最终端到端SVS系统的最佳合成质量。利用Mpop 600中文演唱语音数据集对该系统进行了测试。实验结果表明，该系统在音乐表现力和高频声学细节方面均优于以往的经典系统。此外，对抗性声学模型稳定收敛，无需强制执行重构目标，表明所提出的DDPM和WGAN组合架构相对于替代的基于GAN的SVS系统具有收敛稳定性。摘要：Singing voice synthesis (SVS) is the computer production of a human-like singing voice from given musical scores. To accomplish end-to-end SVS effectively and efficiently, this work adopts the acoustic model-neural vocoder architecture established for high-quality speech and singing voice synthesis. Specifically, this work aims to pursue a higher level of expressiveness in synthesized voices by combining the diffusion denoising probabilistic model (DDPM) and \emph{Wasserstein} generative adversarial network (WGAN) to construct the backbone of the acoustic model. On top of the proposed acoustic model, a HiFi-GAN neural vocoder is adopted with integrated fine-tuning to ensure optimal synthesis quality for the resulting end-to-end SVS system. This end-to-end system was evaluated with the multi-singer Mpop600 Mandarin singing voice dataset. In the experiments, the proposed system exhibits improvements over previous landmark counterparts in terms of musical expressiveness and high-frequency acoustic details. Moreover, the adversarial acoustic model converged stably without the need to enforce reconstruction objectives, indicating the convergence stability of the proposed DDPM and WGAN combined architecture over alternative GAN-based SVS systems.
【7】 The ReturnZero System for VoxCeleb Speaker Recognition Challenge 2022
标题：VoxCeleb说话人识别挑战赛2022的ReturnZero系统
链接：https://arxiv.org/abs/2209.10147
作者：Sangwon Suh,Sunjong Park机构：Return Zero Inc., Seoul, Korea备注：4 pages, 4 tables, technical report摘要：在本文中，我们描述了团队RTZR VoxCeleb说话人识别挑战2022（VoxSRC-22）在封闭数据集中得分最高的提交，说话人验证轨道1。表现最好的系统是7个模型的融合，其中包含3种不同类型的模型架构。我们专注于训练模型来学习额外的时间信息。因此，对于每个发声，用4-6秒帧训练所有模型。此外，我们还应用了大裕度微调策略，该策略在我们的一些融合模型的先前挑战中表现出了良好的性能。在评价过程中，采用了自适应对称归一化（AS-Norm）和矩阵评分平均（MSA）两种评分方法。最后，我们将模型与逻辑回归混合，以融合所有训练好的模型。最终提交的VoxSRC 22测试集实现了0.165 DCF和2.912% EER。摘要：In this paper, we describe the top-scoring submissions for team RTZR VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22) in the closed dataset, speaker verification Track 1. The top performed system is a fusion of 7 models, which contains 3 different types of model architectures. We focus on training models to learn extra-temporal information. Therefore, all models were trained with 4-6 second frames for each utterance. Also, we apply the Large Margin Fine-tuning strategy which has shown good performance on the previous challenges for some of our fusion models. While the evaluation process, we apply the scoring methods with adaptive symmetric normalization (AS-Norm) and matrix score average (MSA). Finally, we mix up models with logistic regression to fuse all the trained models. The final submission achieves 0.165 DCF and 2.912% EER on the VoxSRC22 test set.
【8】 Boosting Star-GANs for Voice Conversion with Contrastive Discriminator
标题：利用对比度判别器增强STAR-GANS语音转换
链接：https://arxiv.org/abs/2209.10088
作者：Shijing Si,Jianzong Wang,Xulong Zhang,Xiaoyang Qu,Ning Cheng,Jing Xiao机构：Ping An Technology (Shenzhen) Co., Ltd., China, School of Economics and Finance, Shanghai International Studies University, China备注：12 pages, 3 figures, Accepted by ICONIP 2022. arXiv admin note: text overlap with arXiv:1907.12279 by other authors摘要：StarGAN-VC等非并行多域语音转换方法已广泛应用于多种场景。然而，由于这些模型复杂的对抗性网络结构，它们的训练通常提出挑战。为了解决这一问题，在本工作中，我们利用了最先进的对比学习技术，并将一个有效的暹罗网络结构纳入StarGAN鉴别器。该方法提高了训练的稳定性，有效地防止了训练过程中的鉴别器过拟合问题。我们在语音转换挑战（VCC 2018）数据集上进行了实验，并进行了用户研究，以验证我们框架的性能。实验结果表明，SimSiam-StarGAN-VC方法在客观和主观两个方面都显著优于现有的StarGAN-VC方法.摘要：Nonparallel multi-domain voice conversion methods such as the StarGAN-VCs have been widely applied in many scenarios. However, the training of these models usually poses a challenge due to their complicated adversarial network architectures. To address this, in this work we leverage the state-of-the-art contrastive learning techniques and incorporate an efficient Siamese network structure into the StarGAN discriminator. Our method is called SimSiam-StarGAN-VC and it boosts the training stability and effectively prevents the discriminator overfitting issue in the training process. We conduct experiments on the Voice Conversion Challenge (VCC 2018) dataset, plus a user study to validate the performance of our framework. Our experimental results show that SimSiam-StarGAN-VC significantly outperforms existing StarGAN-VC methods in terms of both the objective and subjective metrics.
【9】 Language-based Audio Retrieval Task in DCASE 2022 Challenge
标题：DCASE 2022挑战赛中基于语言的音频检索任务
链接：https://arxiv.org/abs/2209.09967
作者：Huang Xie,Samuel Lipping,Tuomas Virtanen机构：Audio Research Group, Tampere University, Tampere, Finland备注：Accepted at DCASE 2022 Workshop. arXiv admin note: substantial text overlap with arXiv:2206.06108摘要：基于语言的音频检索是一种任务，其中自然语言文本字幕被用作查询以从数据集检索音频信号。它被首次引入DCASE 2022 Challenge作为任务6的子任务6B，其目的是开发计算系统来模拟音频信号和自由形式的文本描述之间的关系。与关于为音频信号生成音频字幕的音频字幕（子任务6A）相比，基于语言的音频检索（子任务6B）关注于根据音频信号与自然语言文本字幕的相关性对音频信号进行排序。在DCASE 2022 Challenge中，为子任务6B提供的基线系统的性能明显优于其他系统，mAP@10的最高性能为0.276。本文介绍了子任务6B在提交的系统性能和分析方面的成果。摘要：Language-based audio retrieval is a task, where natural language textual captions are used as queries to retrieve audio signals from a dataset. It has been first introduced into DCASE 2022 Challenge as Subtask 6B of task 6, which aims at developing computational systems to model relationships between audio signals and free-form textual descriptions. Compared with audio captioning (Subtask 6A), which is about generating audio captions for audio signals, language-based audio retrieval (Subtask 6B) focuses on ranking audio signals according to their relevance to natural language textual captions. In DCASE 2022 Challenge, the provided baseline system for Subtask 6B was significantly outperformed, with top performance being 0.276 in mAP@10. This paper presents the outcome of Subtask 6B in terms of submitted systems' performance and analysis.
eess.AS音频处理
【1】 An Initial study on Birdsong Re-synthesis Using Neural Vocoders
标题：用神经声码器进行BirdSong再合成的初步研究
链接：https://arxiv.org/abs/2209.10479
* 与cs.SD语音【5】为同一篇
作者：Rhythm Bhatia,Tomi H. Kinnunen机构：University of Eastern Finland备注：To appear in 24th International Conference on Speech and Computer (SPECOM), GURUGRAM, INDIA摘要：现代语音合成使用神经声码器来直接对原始波形样本进行建模。这种增加的多功能性已经将声码器的范围从语音扩展到诸如音乐的其它领域。我们解决另一个有趣的领域的生物声学。我们提供了使用传统（WORLD）和两种神经（WaveNet自动编码器，并行WaveGAN）声码器的鸟鸣声的初步比较分析-再合成实验。我们的主观结果表明三种声码器在物种辨别方面没有差异（ABX测试）。尽管如此，WORLD声码器样本在保持鸟样品质方面被评定为较高（MOS测试）。所有的声码器都面临着音高和发声的问题。我们的结果表明了在处理低质量野生动物音频数据时的一些挑战。摘要：Modern speech synthesis uses neural vocoders to model raw waveform samples directly. This increased versatility has expanded the scope of vocoders from speech to other domains, such as music. We address another interesting domain of bio-acoustics. We provide initial comparative analysis-resynthesis experiments of birdsong using traditional (WORLD) and two neural (WaveNet autoencoder, parallel WaveGAN) vocoders. Our subjective results indicate no difference in the three vocoders in terms of species discrimination (ABX test). Nonetheless, the WORLD vocoder samples were rated higher in terms of retaining bird-like qualities (MOS test). All vocoders faced issues with pitch and voicing. Our results indicate some of the challenges in processing low-quality wildlife audio data.
【2】 Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN
标题：基于扩散概率瓦瑟斯坦去噪的普通话歌声合成
链接：https://arxiv.org/abs/2209.10446
* 与cs.SD语音【6】为同一篇
作者：Yin-Ping Cho,Yu Tsao,Hsin-Min Wang,Yi-Wen Liu机构：National Tsing Hua University, Hsinchu, Taiwan, Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan, Institute of Information Science, Academia Sinica, Taipei, Taiwan摘要：歌唱声音合成（SVS）是从给定的乐谱中用计算机产生类似人类的歌唱声音。为了有效地实现端到端的SVS，本文采用了为高质量语音和歌唱声合成而建立的声学模型-神经元声码器结构。具体而言，本工作旨在通过将扩散去噪概率模型（DDPM）与\emph{Wasserstein}生成对抗网络（WGAN）相结合来构建声学模型的主干，从而在合成语音中追求更高级别的表现力。在所提出的声学模型之上，采用了具有集成微调的HiFi-GAN神经声码器，以确保最终端到端SVS系统的最佳合成质量。利用Mpop 600中文演唱语音数据集对该系统进行了测试。实验结果表明，该系统在音乐表现力和高频声学细节方面均优于以往的经典系统。此外，对抗性声学模型稳定收敛，无需强制执行重构目标，表明所提出的DDPM和WGAN组合架构相对于替代的基于GAN的SVS系统具有收敛稳定性。摘要：Singing voice synthesis (SVS) is the computer production of a human-like singing voice from given musical scores. To accomplish end-to-end SVS effectively and efficiently, this work adopts the acoustic model-neural vocoder architecture established for high-quality speech and singing voice synthesis. Specifically, this work aims to pursue a higher level of expressiveness in synthesized voices by combining the diffusion denoising probabilistic model (DDPM) and \emph{Wasserstein} generative adversarial network (WGAN) to construct the backbone of the acoustic model. On top of the proposed acoustic model, a HiFi-GAN neural vocoder is adopted with integrated fine-tuning to ensure optimal synthesis quality for the resulting end-to-end SVS system. This end-to-end system was evaluated with the multi-singer Mpop600 Mandarin singing voice dataset. In the experiments, the proposed system exhibits improvements over previous landmark counterparts in terms of musical expressiveness and high-frequency acoustic details. Moreover, the adversarial acoustic model converged stably without the need to enforce reconstruction objectives, indicating the convergence stability of the proposed DDPM and WGAN combined architecture over alternative GAN-based SVS systems.
【3】 GIST-AiTeR System for the Diarization Task of the 2022 VoxCeleb Speaker Recognition Challenge
标题：2022年VoxCeleb说话人识别挑战赛二值化任务的GIST-AITER系统
链接：https://arxiv.org/abs/2209.10357
作者：Dongkeon Park,Yechan Yu,Kyeong Wan Park,Ji Won Kim,Hong Kook Kim机构：AI Graduate School,School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju , Republic of Korea摘要：本报告描述了GIST-AiTeR团队在2022年VoxCeleb说话人识别挑战赛（VoxSRC）第4轮比赛中的提交系统。该系统主要包括语音增强、语音激活检测、多尺度说话人嵌入、基于概率线性判别分析的说话人聚类和重叠语音检测模型。我们首先根据不同的模型组合，以最佳的实验努力，建构四种不同的二值化系统。我们的最终提交是所有4个系统的集成系统，在挑战评估集上获得了5.12%的二值化错误率，在挑战的二值化轨道上排名第三。摘要：This report describes the submission system of the GIST-AiTeR team at the 2022 VoxCeleb Speaker Recognition Challenge (VoxSRC) Track 4. Our system mainly includes speech enhancement, voice activity detection , multi-scaled speaker embedding, probabilistic linear discriminant analysis-based speaker clustering, and overlapped speech detection models. We first construct four different diarization systems according to different model combinations with the best experimental efforts. Our final submission is an ensemble system of all the four systems and achieves a diarization error rate of 5.12\% on the challenge evaluation set, ranked third at the diarization track of the challenge.
【4】 The ReturnZero System for VoxCeleb Speaker Recognition Challenge 2022
标题：VoxCeleb说话人识别挑战赛2022的ReturnZero系统
链接：https://arxiv.org/abs/2209.10147
* 与cs.SD语音【7】为同一篇
作者：Sangwon Suh,Sunjong Park机构：Return Zero Inc., Seoul, Korea备注：4 pages, 4 tables, technical report摘要：在本文中，我们描述了团队RTZR VoxCeleb说话人识别挑战2022（VoxSRC-22）在封闭数据集中得分最高的提交，说话人验证轨道1。表现最好的系统是7个模型的融合，其中包含3种不同类型的模型架构。我们专注于训练模型来学习额外的时间信息。因此，对于每个发声，用4-6秒帧训练所有模型。此外，我们还应用了大裕度微调策略，该策略在我们的一些融合模型的先前挑战中表现出了良好的性能。在评价过程中，采用了自适应对称归一化（AS-Norm）和矩阵评分平均（MSA）两种评分方法。最后，我们将模型与逻辑回归混合，以融合所有训练好的模型。最终提交的VoxSRC 22测试集实现了0.165 DCF和2.912% EER。摘要：In this paper, we describe the top-scoring submissions for team RTZR VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22) in the closed dataset, speaker verification Track 1. The top performed system is a fusion of 7 models, which contains 3 different types of model architectures. We focus on training models to learn extra-temporal information. Therefore, all models were trained with 4-6 second frames for each utterance. Also, we apply the Large Margin Fine-tuning strategy which has shown good performance on the previous challenges for some of our fusion models. While the evaluation process, we apply the scoring methods with adaptive symmetric normalization (AS-Norm) and matrix score average (MSA). Finally, we mix up models with logistic regression to fuse all the trained models. The final submission achieves 0.165 DCF and 2.912% EER on the VoxSRC22 test set.
【5】 Boosting Star-GANs for Voice Conversion with Contrastive Discriminator
标题：利用对比度判别器增强STAR-GANS语音转换
链接：https://arxiv.org/abs/2209.10088
* 与cs.SD语音【8】为同一篇
作者：Shijing Si,Jianzong Wang,Xulong Zhang,Xiaoyang Qu,Ning Cheng,Jing Xiao机构：Ping An Technology (Shenzhen) Co., Ltd., China, School of Economics and Finance, Shanghai International Studies University, China备注：12 pages, 3 figures, Accepted by ICONIP 2022. arXiv admin note: text overlap with arXiv:1907.12279 by other authors摘要：StarGAN-VC等非并行多域语音转换方法已广泛应用于多种场景。然而，由于这些模型复杂的对抗性网络结构，它们的训练通常提出挑战。为了解决这一问题，在本工作中，我们利用了最先进的对比学习技术，并将一个有效的暹罗网络结构纳入StarGAN鉴别器。该方法提高了训练的稳定性，有效地防止了训练过程中的鉴别器过拟合问题。我们在语音转换挑战（VCC 2018）数据集上进行了实验，并进行了用户研究，以验证我们框架的性能。实验结果表明，SimSiam-StarGAN-VC方法在客观和主观两个方面都显著优于现有的StarGAN-VC方法.摘要：Nonparallel multi-domain voice conversion methods such as the StarGAN-VCs have been widely applied in many scenarios. However, the training of these models usually poses a challenge due to their complicated adversarial network architectures. To address this, in this work we leverage the state-of-the-art contrastive learning techniques and incorporate an efficient Siamese network structure into the StarGAN discriminator. Our method is called SimSiam-StarGAN-VC and it boosts the training stability and effectively prevents the discriminator overfitting issue in the training process. We conduct experiments on the Voice Conversion Challenge (VCC 2018) dataset, plus a user study to validate the performance of our framework. Our experimental results show that SimSiam-StarGAN-VC significantly outperforms existing StarGAN-VC methods in terms of both the objective and subjective metrics.
【6】 Language-based Audio Retrieval Task in DCASE 2022 Challenge
标题：DCASE 2022挑战赛中基于语言的音频检索任务
链接：https://arxiv.org/abs/2209.09967
* 与cs.SD语音【9】为同一篇
作者：Huang Xie,Samuel Lipping,Tuomas Virtanen机构：Audio Research Group, Tampere University, Tampere, Finland备注：Accepted at DCASE 2022 Workshop. arXiv admin note: substantial text overlap with arXiv:2206.06108摘要：基于语言的音频检索是一种任务，其中自然语言文本字幕被用作查询以从数据集检索音频信号。它被首次引入DCASE 2022 Challenge作为任务6的子任务6B，其目的是开发计算系统来模拟音频信号和自由形式的文本描述之间的关系。与关于为音频信号生成音频字幕的音频字幕（子任务6A）相比，基于语言的音频检索（子任务6B）关注于根据音频信号与自然语言文本字幕的相关性对音频信号进行排序。在DCASE 2022 Challenge中，为子任务6B提供的基线系统的性能明显优于其他系统，mAP@10的最高性能为0.276。本文介绍了子任务6B在提交的系统性能和分析方面的成果。摘要：Language-based audio retrieval is a task, where natural language textual captions are used as queries to retrieve audio signals from a dataset. It has been first introduced into DCASE 2022 Challenge as Subtask 6B of task 6, which aims at developing computational systems to model relationships between audio signals and free-form textual descriptions. Compared with audio captioning (Subtask 6A), which is about generating audio captions for audio signals, language-based audio retrieval (Subtask 6B) focuses on ranking audio signals according to their relevance to natural language textual captions. In DCASE 2022 Challenge, the provided baseline system for Subtask 6B was significantly outperformed, with top performance being 0.276 in mAP@10. This paper presents the outcome of Subtask 6B in terms of submitted systems' performance and analysis.
【7】 Learning Hierarchical Metrical Structure Beyond Measures
标题：学习超越度量的层次化度量结构
链接：https://arxiv.org/abs/2209.10259
* 与cs.SD语音【1】为同一篇
作者：Junyan Jiang,Daniel Chin,Yixiao Zhang,Gus Xia机构：Music X Lab, NYU Shanghai, MBZUAI, Centre for Digital Music, QMUL备注：Accepted at the International Society for Music Information Retrieval (ISMIR), 2022摘要：音乐包含着超越节拍和小节的层次结构。层次结构的音乐标注在音乐信息检索和计算机音乐学中具有重要的应用价值，但在现有的数字音乐数据库中却很少见到。本文提出了一种基于数据驱动的自动抽取韵律层次结构的方法。提出了一种基于时间卷积网络-条件随机场（TCN-CRF）结构的新模型。给定一个符号乐谱，我们的模型以节拍量化的形式接收任意数量的声部，并预测出从下拍级到小节级的4级分层韵律结构。我们还使用RWC-POP MIDI文件注释数据集，以便于训练和评估。实验结果表明，在不同的编排设置下，该方法的性能优于基于规则的方法。我们还对模型预测进行了一些简单的音乐学分析。所有的演示、数据集和预先训练的模型都可以在Github上公开获得。摘要：Music contains hierarchical structures beyond beats and measures. While hierarchical structure annotations are helpful for music information retrieval and computer musicology, such annotations are scarce in current digital music databases. In this paper, we explore a data-driven approach to automatically extract hierarchical metrical structures from scores. We propose a new model with a Temporal Convolutional Network-Conditional Random Field (TCN-CRF) architecture. Given a symbolic music score, our model takes in an arbitrary number of voices in a beat-quantized form, and predicts a 4-level hierarchical metrical structure from downbeat-level to section-level. We also annotate a dataset using RWC-POP MIDI files to facilitate training and evaluation. We show by experiments that the proposed method performs better than the rule-based approach under different orchestration settings. We also perform some simple musicological analysis on the model predictions. All demos, datasets and pre-trained models are publicly available on Github.
【8】 Dynamic Time-Alignment of Dimensional Annotations of Emotion using Recurrent Neural Networks
标题：基于递归神经网络的情感维度标注的动态时间对齐
链接：https://arxiv.org/abs/2209.10223
* 与cs.SD语音【2】为同一篇
作者：Sina Alisamir,Fabien Ringeval,Francois Portet机构：Atos & LIG, Grenoble, France, Grenoble INP, LIG, Univ. Grenoble Alpes, Inria, CNRS, François Portet摘要：大多数自动情感识别系统利用情感的时间连续注释来提供如在现实生活交互中观察到的自发表情的细粒度描述。由于情绪是相当主观的，它的注释通常由几个注释者执行，他们提供给定维度的轨迹，即描述诸如唤醒或效价等维度的时间连续序列。然而，同一表情的注释在注释器之间很少在时间或值上一致，这在用于学习情感的预测模型的轨迹中增加了偏差和延迟。因此，我们提出了一种方法，可以动态补偿注释之间的不一致性，并使用递归神经网络将轨迹与相应的声学特征同步。实验评估是在几个情感数据集上进行的，这些数据集包括中国、法国、德国和匈牙利的参与者，他们在无噪声条件下或在野外进行远程交互。实验结果表明，该方法可以显著提高标注者之间的一致性，以及轨迹和音频特征之间的相关性，无论是唤醒还是效价。此外，使用简单的轻量模型在这些维度的自动预测中获得了改进，特别是对于无噪声条件下的效价和在野外捕获的记录的唤醒。摘要：Most automatic emotion recognition systems exploit time-continuous annotations of emotion to provide fine-grained descriptions of spontaneous expressions as observed in real-life interactions. As emotion is rather subjective, its annotation is usually performed by several annotators who provide a trace for a given dimension, i.e. a time-continuous series describing a dimension such as arousal or valence. However, annotations of the same expression are rarely consistent between annotators, either in time or in value, which adds bias and delay in the trace that is used to learn predictive models of emotion. We therefore propose a method that can dynamically compensate inconsistencies across annotations and synchronise the traces with the corresponding acoustic features using Recurrent Neural Networks. Experimental evaluations were carried on several emotion data sets that include Chinese, French, German, and Hungarian participants who interacted remotely in either noise-free conditions or in-the-wild. The results show that our method can significantly increase inter-annotator agreement, as well as correlation between traces and audio features, for both arousal and valence. In addition, improvements are obtained in the automatic prediction of these dimensions using simple light-weight models, especially for valence in noise-free conditions, and arousal for recordings captured in-the-wild.
【9】 Setting the rhythm scene: deep learning-based drum loop generation from arbitrary language cues
标题：设置节奏场景：基于深度学习的任意语言提示的鼓循环生成
链接：https://arxiv.org/abs/2209.10016
* 与cs.SD语音【3】为同一篇
机构：Independent Researcher摘要：生成式人工智能模型可以成为音乐创作和现场表演的宝贵辅助工具，既可以帮助专业音乐家，也可以帮助业余爱好者实现音乐创作过程的民主化。在此，我们提出一种新颖的方法，给定一个英语单词或短语，生成体现给定语言线索的“情绪”的4件鼓图案的2个罗盘，或者可以用于由语言线索描述的视听场景。我们将此工具设想为电子音乐和视听配乐制作的作曲辅助工具，或现场表演的即兴创作工具。为了产生该模型的训练样本，除了人工标注“场景”或“情绪”术语之外，我们还设计了一种新颖的方法来提取任何歌曲的共识鼓点。这包括一个2小节，4件鼓模式，代表了一首歌曲的主要打击乐主题，可以导入到任何音乐循环设备或现场循环软件。这两个关键组件（从可归纳的输入生成鼓模式和一致打击乐提取）为计算机辅助作曲提供了一种新的方法，并为更全面的节奏生成提供了垫脚石。摘要：Generative artificial intelligence models can be a valuable aid to music composition and live performance, both to aid the professional musician and to help democratize the music creation process for hobbyists. Here we present a novel method that, given an English word or phrase, generates 2 compasses of a 4-piece drum pattern that embodies the "mood" of the given language cue, or that could be used for an audiovisual scene described by the language cue. We envision this tool as composition aid for electronic music and audiovisual soundtrack production, or an improvisation tool for live performance. In order to produce the training samples for this model, besides manual annotation of the "scene" or "mood" terms, we have designed a novel method to extract the consensus drum track of any song. This consists of a 2-bar, 4-piece drum pattern that represents the main percussive motif of a song, which could be imported into any music loop device or live looping software. These two key components (drum pattern generation from a generalizable input, and consensus percussion extraction) present a novel approach to computer-aided composition and provide a stepping stone for more comprehensive rhythm generation.
【10】 Meta-Learning for Adaptive Filters with Higher-Order Frequency Dependencies
标题：高阶频率相关自适应滤波器的元学习
链接：https://arxiv.org/abs/2209.09955
* 与cs.SD语音【4】为同一篇
作者：Junkai Wu,Jonah Casebeer,Nicholas J. Bryan,Paris Smaragdis机构：♯ University of Illinois at Urbana-Champaign, ♭ Adobe Research备注：Source code and audio examples: this https URL摘要：自适应滤波器适用于许多信号处理任务，包括声学回声消除、波束形成等。自适应滤波器通常使用诸如最小均方（LMS）、递归最小二乘（RLS）或卡尔曼滤波器更新之类的算法来控制。为了简单起见，这种模型通常应用在频域中，假设频率独立处理，并且不利用高阶频率依赖性。然而，最近关于元自适应滤波器的工作已经表明，我们可以使用神经网络来控制滤波器自适应，而无需人工推导，这激发了利用这种信息的新工作。在本文中，我们提出高阶次自适应滤波器，这是对次自适应滤波器的一个关键改进，它结合了高阶频率相关性。我们演示了我们的声学回声消除方法，并开发了一系列滤波器，与竞争对手的基线相比，这些滤波器可产生数dB的改进，并且复杂性至少降低了一个数量级。此外，我们显示了我们的改进在有或没有下游语音增强器的情况下都成立。摘要：Adaptive filters are applicable to many signal processing tasks including acoustic echo cancellation, beamforming, and more. Adaptive filters are typically controlled using algorithms such as least-mean squares(LMS), recursive least squares(RLS), or Kalman filter updates. Such models are often applied in the frequency domain, assume frequency independent processing, and do not exploit higher-order frequency dependencies, for simplicity. Recent work on meta-adaptive filters, however, has shown that we can control filter adaptation using neural networks without manual derivation, motivating new work to exploit such information. In this work, we present higher-order meta-adaptive filters, a key improvement to meta-adaptive filters that incorporates higher-order frequency dependencies. We demonstrate our approach on acoustic echo cancellation and develop a family of filters that yield multi-dB improvements over competitive baselines, and are at least an order-of-magnitude less complex. Moreover, we show our improvements hold with or without a downstream speech enhancer.
机器翻译，仅供参考
每日语音速递