今天跟大家分享一篇语音相关的论文合集：cs.SD语音13篇，eess.AS音频处理14篇。
cs.SD语音
【1】 FAD: A Chinese Dataset for Fake Audio Detection
标题： FAD：一个用于虚假音频检测的中文数据集
链接：https://arxiv.org/abs/2207.12308
作者：Haoxin Ma,Jiangyan Yi,Chenglong Wang,Xinrui Yan,Jianhua Tao,Tao Wang,Shiming Wang,Le Xu,Ruibo Fu机构：NLPR, Institute of Automation, Chinese Academy of Sciences, China, School of Artificial Intelligence, University of Chinese Academy of Sciences, China, CAS Center for Excellence in Brain Science and Intelligence Technology, China备注：submitted to NeurIPS 2022 Track on Datasets and Benchmarks摘要：虚假音频检测是一个日益受到关注的问题，已经设计了一些相关的数据集用于研究，但是在加性噪声条件下，还没有一个标准的公开的中文数据集，本文旨在填补这一空白，设计一个中文虚假音频检测数据集（FAD）来研究更一般的检测方法.使用12种主流的语音产生技术来产生假音频.为了模拟真实的场景，在5种不同的信噪比下，选择3个噪声数据集进行加噪. FAD数据集不仅可以用于虚假音频检测，而且可以用于音频取证中的虚假话语检测算法.给出了基线结果并进行了分析.结果表明，具有泛化能力的虚假音频检测方法仍然具有挑战性. FAD数据集是公开的.摘要：Fake audio detection is a growing concern and some relevant datasets have been designed for research. But there is no standard public Chinese dataset under additive noise conditions. In this paper, we aim to fill in the gap and design a Chinese fake audio detection dataset (FAD) for studying more generalized detection methods. Twelve mainstream speech generation techniques are used to generate fake audios. To simulate the real-life scenarios, three noise datasets are selected for noisy adding at five different signal noise ratios. FAD dataset can be used not only for fake audio detection, but also for detecting the algorithms of fake utterances for audio forensics. Baseline results are presented with analysis. The results that show fake audio detection methods with generalization remain challenging. The FAD dataset is publicly available.
【2】 Domain Adapting Speech Emotion Recognition modals to real-world scenario with Deep Reinforcement Learning
标题： 基于深度强化学习的领域自适应语音情感识别模型
链接：https://arxiv.org/abs/2207.12248
作者：Thejan Rajapakshe,Rajib Rana,Sara Khalifa机构： Rajapakshe is affiliated with University of Southern Queensland, Rana is with University of Southern Queensland摘要：随着深度学习在机器学习领域的广泛应用，深度强化学习已经成为一种流行的训练范式.领域自适应允许我们在训练一个阶段后跨领域转移由模型学习到的知识.现有领域自适应算法的缺点之一是不能使现有模型适应真实世界的领域.本文提出了一种深度强化学习算法———————————-本文提出了一种基于自适应策略的语音情感识别方法，该方法可以在与环境交互的同时，不断地收集反馈信息，使预先训练好的模型适应新的领域，并将该方法应用于语音情感识别任务中，我们的方法在跨语料库和跨语言模式上分别比监督学习策略的性能高出42%和20%。摘要：Deep reinforcement learning has been a popular training paradigm as deep learning has gained popularity in the field of machine learning. Domain adaptation allows us to transfer knowledge learnt by a model across domains after a phase of training. The inability to adapt an existing model to a real-world domain is one of the shortcomings of current domain adaptation algorithms. We present a deep reinforcement learning-based strategy for adapting a pre-trained model to a newer domain while interacting with the environment and collecting continual feedback. This method was used on the Speech Emotion Recognition task, which included both cross-corpus and cross-language domain adaption schema. Furthermore, it demonstrates that in a real-world environment, our approach outperforms the supervised learning strategy by 42% and 20% in cross-corpus and cross-language schema, respectively.
【3】 Cross-Modal Contrastive Representation Learning for Audio-to-Image Generation
标题： 基于跨通道对比表示学习的音像转换算法
链接：https://arxiv.org/abs/2207.12121
作者：HaeChun Chung,JooYong Shim,Jong-Kook Kim机构：Dept. of Electrical Engineering, Korea University备注：7 pages, 3 figures, Accepted to MUE 2022摘要：多模态信息提供了对信息的多种视角，可以提高对信息的理解，因此，从已有的数据中生成不同模态的数据对于提高理解可能是至关重要的.本文提出了一种基于多模态数据的信息检索方法，我们研究了跨通道音频到图像的生成问题，并提出了跨通道对比表征学习实验结果表明，CMCRL算法能够从音频中提取有用的特征，并将其应用于图像生成阶段，提高了生成图像的质量。摘要：Multiple modalities for certain information provide a variety of perspectives on that information, which can improve the understanding of the information. Thus, it may be crucial to generate data of different modality from the existing data to enhance the understanding. In this paper, we investigate the cross-modal audio-to-image generation problem and propose Cross-Modal Contrastive Representation Learning (CMCRL) to extract useful features from audios and use it in the generation phase. Experimental results show that CMCRL enhances quality of images generated than previous research.
【4】 Source Separation of Unknown Numbers of Single-Channel Underwater Acoustic Signals Based on Autoencoders
标题： 基于自编码器的未知数目单通道水声信号源分离
链接：https://arxiv.org/abs/2207.11749
作者：Qinggang Sun,Kejun Wang机构：Q. Sun and K. Wang are with the College of Intelligent Systems Science and, Engineering, Harbin Engineering University, Harbin, Heilongjiang, China, and, K. Wang is also with the School of Information Technology, Beijing Institute, Zhuhai, Guangdong, (e-mails:备注：6 pages, 2 figures,3 tables. For codes, see this https URL This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible摘要：单通道水声信号的分离是一个具有实际意义的难题，针对未知个数的信号分离问题，提出了一种输出通道数固定的信号分离方法，避免了由于输出与目标对准而引起的排列问题所带来的维数灾难，具体而言，我们修改了基于自动编码器为已知数量的信号开发的两种算法，并提出了一种新的静音信道下的性能评估方法。在模拟的舰船辐射噪声混合信号上的实验表明，该方法可以获得与已知信号数时相似的分离性能。静音通道输出也不错。摘要：The separation of single-channel underwater acoustic signals is a challenging problem with practical significance. In view of the signal separation problem with unknown numbers of signals, we propose a solution with a fixed number of output channels, enabling it to avoid the dimensional disaster caused by the permutation problem induced by the alignment of outputs to targets. Specifically, we modify two algorithms developed for known numbers of signals based on autoencoders, which are highly explainable. We also propose a new performance evaluation method for situations with mute channels. Experiments conducted on simulated mixtures of radiated ship noise show that the proposed solution can achieve similar separation performance to that attained with a known number of signals. The mute channel output is also good.
【5】 HouseX: A Fine-grained House Music Dataset and its Potential in the Music Industry
标题： HouseX：一个细粒度的House音乐数据集及其在音乐产业中的潜力
链接：https://arxiv.org/abs/2207.11690
机构：∗ New York University Shanghai, Shanghai, China摘要：机器声音分类一直是音乐技术的基本任务之一.声音分类的一个主要分支是音乐流派的分类.然而，尽管现有的音乐流派数据集涵盖了大多数音乐流派，但往往不包含细粒度的标签来指示音乐的详细子流派.考虑到混音带或DJ中歌曲流派的一致性，提出了一种基于机器声音分类的音乐流派分类方法.该方法基于机器声音分类的基本思想，利用机器声音分类的基本思想，提出了一种基于机器声音分类的音乐流派分类方法.该方法基于机器声音分类的基本思想，提出了一种基于机器声音分类的音乐流派分类方法本文在现场音乐背景下，收集了一个家庭音乐数据集，并对该数据集进行了标注，给出了未来家庭音乐、低音家庭音乐、渐进家庭音乐和旋律家庭音乐4个子类型的标注.实验表明，我们的标注很好地体现了不同类型的特点.同时，我们还建立了基于旋律谱图的子类型分类基线模型，此外，我们还提出了我们的数据集和基准模型的几个应用场景，其中一个模拟的科幻隧道作为一个简短的演示，在3D建模软件中构建和渲染，灯光的颜色由我们的模型的输出自动化。摘要：Machine sound classification has been one of the fundamental tasks of music technology. A major branch of sound classification is the classification of music genres. However, though covering most genres of music, existing music genre datasets often do not contain fine-grained labels that indicate the detailed sub-genres of music. In consideration of the consistency of genres of songs in a mixtape or in a DJ (live) set, we have collected and annotated a dataset of house music that provide 4 sub-genre labels, namely future house, bass house, progressive house and melodic house. Experiments show that our annotations well exhibit the characteristics of different categories. Also, we have built baseline models that classify the sub-genre based on the mel-spectrograms of a track, achieving strongly competitive results. Besides, we have put forward a few application scenarios of our dataset and baseline model, with a simulated sci-fi tunnel as a short demo built and rendered in a 3D modeling software, with the colors of the lights automated by the output of our model.
【6】 Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparities
标题： 语音识别中的公平性：发现和缓解性能差异
链接：https://arxiv.org/abs/2207.11345
作者：Pranav Dheram,Murugesan Ramakrishnan,Anirudh Raju,I-Fan Chen,Brian King,Katherine Powell,Melissa Saboowala,Karan Shetty,Andreas Stolcke备注：Proc. Interspeech 2022摘要：至于其他形式的人工智能，最近已经就不同用户群体之间的性能差异对语音识别进行了研究。（1）识别遭受低于标准表现的说话者群组，以及（2）针对发现的群组应用公平性缓解措施，我们报告了使用来自产品级AI辅助语音识别系统的数据发现和减轻性能差异的初步发现。我们将基于地理和人口统计信息的群组发现与使用说话人嵌入技术的更可扩展的无人类标签的说话人分组方法进行比较。我们发现，对代表不足的群组进行过采样以及通过附加输入变量对说话者群组成员资格进行建模，减小了表现最好和最差的群组之间的差距，而不会使整体识别准确度恶化。摘要：As for other forms of AI, speech recognition has recently been examined with respect to performance disparities across different user cohorts. One approach to achieve fairness in speech recognition is to (1) identify speaker cohorts that suffer from subpar performance and (2) apply fairness mitigation measures targeting the cohorts discovered. In this paper, we report on initial findings with both discovery and mitigation of performance disparities using data from a product-scale AI assistant speech recognition system. We compare cohort discovery based on geographic and demographic information to a more scalable method that groups speakers without human labels, using speaker embedding technology. For fairness mitigation, we find that oversampling of underrepresented cohorts, as well as modeling speaker cohort membership by additional input variables, reduces the gap between top- and bottom-performing cohorts, without deteriorating overall recognition accuracy.
【7】 Transplantation of Conversational Speaking Style with Interjections in Sequence-to-Sequence Speech Synthesis
标题： 序列到序列语音合成中带感叹词的会话说话风格移植
链接：https://arxiv.org/abs/2207.12262
作者：Raul Fernandez,David Haws,Guy Lorberbom,Slava Shechtman,Alexander Sorin备注：Accepted for presentation at Interspeech 2022摘要：已知直接从语音序列生成低级声学特征的序列到序列文本到语音体系结构在被提供有足够量的训练数据时产生自然的和有表达力的语音。（在多风格多扬声器设置中），这对于创建可扩展的、可定制的人机交互系统是非常需要的。本文以说话人会话语料库为研究对象，分析了语料库的设计方法，并探讨了在语音转换数据增强的辅助下进行风格迁移的可行性。在一组主观听力实验中，该方法在不降低质量的前提下实现了高保真的风格迁移，但也观察到了一定的语音角色转换，需要进一步改进语音转换。摘要：Sequence-to-Sequence Text-to-Speech architectures that directly generate low level acoustic features from phonetic sequences are known to produce natural and expressive speech when provided with adequate amounts of training data. Such systems can learn and transfer desired speaking styles from one seen speaker to another (in multi-style multi-speaker settings), which is highly desirable for creating scalable and customizable Human-Computer Interaction systems. In this work we explore one-to-many style transfer from a dedicated single-speaker conversational corpus with style nuances and interjections. We elaborate on the corpus design and explore the feasibility of such style transfer when assisted with Voice-Conversion-based data augmentation. In a set of subjective listening experiments, this approach resulted in high-fidelity style transfer with no quality degradation. However, a certain voice persona shift was observed, requiring further improvements in voice conversion.
【8】 Label Uncertainty Modeling and Prediction for Speech Emotion Recognition using t-Distributions
标题： 基于t分布的语音情感识别标签不确定性建模与预测
链接：https://arxiv.org/abs/2207.12135
作者：Navin Raj Prabhu,Nale Lehmann-Willenbrock,Timo Gerkmann机构：Signal Processing, Universit¨at Hamburg, Hamburg, Germany, Industrial and Organizational Psychology备注：ACCEPTED to ACII 2022 -10th INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING & INTELLIGENT INTERACTION摘要：由于不同的人对其他人的情绪表达的感知不同，因此他们在唤醒和效价方面的注释本身是主观的。为了解决这个问题，这些情绪注释通常由多个注释者收集，并在注释者之间进行平均，以便获得唤醒和效价的标签。然而，除了平均值之外，标签的不确定性也是感兴趣的，并且还应当被建模和预测以用于自动情感识别。在文献中，为了简单起见，通常利用对所收集的注释的高斯假设来进行标签不确定性建模。然而，由于注释器的数量由于资源限制而通常相当小，我们认为高斯方法是一个相当粗糙的假设.相反，在这项工作中，我们建议使用Student's t-分布来建模标签分布，我们推导出相应的基于Kullback-Leibler散度的损失函数并使用它来训练情感标签分布的估计器，通过定性和定量分析，我们在AVEC'16数据集上验证了我们提出的方法。结果表明，我们基于t分布的方法优于高斯方法，具有状态—时间—空间的特性。不确定性建模导致基于语音的情感识别以及最优的甚至更快的收敛。摘要：As different people perceive others' emotional expressions differently, their annotation in terms of arousal and valence are per se subjective. To address this, these emotion annotations are typically collected by multiple annotators and averaged across annotators in order to obtain labels for arousal and valence. However, besides the average, also the uncertainty of a label is of interest, and should also be modeled and predicted for automatic emotion recognition. In the literature, for simplicity, label uncertainty modeling is commonly approached with a Gaussian assumption on the collected annotations. However, as the number of annotators is typically rather small due to resource constraints, we argue that the Gaussian approach is a rather crude assumption. In contrast, in this work we propose to model the label distribution using a Student's t-distribution which allows us to account for the number of annotations available. With this model, we derive the corresponding Kullback-Leibler divergence based loss function and use it to train an estimator for the distribution of emotion labels, from which the mean and uncertainty can be inferred. Through qualitative and quantitative analysis, we show the benefits of the t-distribution over a Gaussian distribution. We validate our proposed method on the AVEC'16 dataset. Results reveal that our t-distribution based approach improves over the Gaussian approach with state-of-the-art uncertainty modeling results in speech-based emotion recognition, along with an optimal and even faster convergence.
【9】 Unsupervised data selection for Speech Recognition with contrastive loss ratios
标题： 基于对比丢失率的语音识别无监督数据选择
链接：https://arxiv.org/abs/2207.12028
作者：Chanho Park,Rehan Ahmad,Thomas Hain机构：Department of Computer Science, University of Sheffield摘要：本文提出了一种基于目标数据集和训练数据集的对比丢失率的子模函数无监督数据选择方法.首先在目标数据集和训练数据集上训练一个使用对比丢失函数的模型.然后，实验结果表明，用该方法选择的训练集训练的模型性能优于基于对数—方差模型的选择方法.由GMM-HMM模型产生的似然性，以单词错误率表示（WER）。当选择固定数量（例如10小时的数据）时，两种方法在Teddalk上的结果之间的差异为20.23%相对WER。该方法也可用于选择数据，目的是使负迁移最小化，同时保持或提高了在整个训练集上训练的模型的性能，结果表明，在WSJCAM0数据集上，当从整个数据集中选取85%时，WER相对降低了6.26%。摘要：This paper proposes an unsupervised data selection method by using a submodular function based on contrastive loss ratios of target and training data sets. A model using a contrastive loss function is trained on both sets. Then the ratio of frame-level losses for each model is used by a submodular function. By using the submodular function, a training set for automatic speech recognition matching the target data set is selected. Experiments show that models trained on the data sets selected by the proposed method outperform the selection method based on log-likelihoods produced by GMM-HMM models, in terms of word error rate (WER). When selecting a fixed amount, e.g. 10 hours of data, the difference between the results of two methods on Tedtalks was 20.23% WER relative. The method can also be used to select data with the aim of minimising negative transfer, while maintaining or improving on performance of models trained on the whole training set. Results show that the WER on the WSJCAM0 data set was reduced by 6.26% relative when selecting 85% from the whole data set.
【10】 ConceptBeam: Concept Driven Target Speech Extraction
标题： 概念梁：概念驱动的目标语音提取
链接：https://arxiv.org/abs/2207.11964
作者：Yasunori Ohishi,Marc Delcroix,Tsubasa Ochiai,Shoko Araki,Daiki Takeuchi,Daisuke Niizumi,Akisato Kimura,Noboru Harada,Kunio Kashino机构：NTT Corporation, Tokyo, Japan备注：Accepted to ACM Multimedia 2022摘要：提出了一种基于语义信息的目标语音提取框架ConceptBeam.目标语音提取是指从混合语音中提取目标说话人的语音.典型的方法是利用音频信号的特性，如谐波结构和到达方向.相比之下，ConceptBeam则利用语义线索来处理这个问题.具体来说，我们提取说话人谈论一个概念的语音，即感兴趣的主题。解决这个新颖的问题将打开通向创新应用的大门，例如专注于在对话中讨论的特定主题的收听系统。与关键字不同，概念是抽象概念，使得直接表示目标概念具有挑战性。在我们的方案中，通过将概念描述符映射到一个共享嵌入空间，将概念编码为语义嵌入，该嵌入空间可以通过深度度量学习的方式使用由图像和其语音字幕组成的成对数据来建立，该方法利用了一组与语音字幕相关联的图像来进行实验，即，我们从这些字幕中产生语音混合，并使用图像或语音信号作为概念描述符，然后我们利用识别片段的声学特征提取目标语音。我们将ConceptBeam与两种方法进行比较：一个基于从识别系统获得的关键词，另一个基于声源分离，我们表明ConceptBeam明显优于基线方法，并有效地提取基于语义表示的语音。摘要：We propose a novel framework for target speech extraction based on semantic information, called ConceptBeam. Target speech extraction means extracting the speech of a target speaker in a mixture. Typical approaches have been exploiting properties of audio signals, such as harmonic structure and direction of arrival. In contrast, ConceptBeam tackles the problem with semantic clues. Specifically, we extract the speech of speakers speaking about a concept, i.e., a topic of interest, using a concept specifier such as an image or speech. Solving this novel problem would open the door to innovative applications such as listening systems that focus on a particular topic discussed in a conversation. Unlike keywords, concepts are abstract notions, making it challenging to directly represent a target concept. In our scheme, a concept is encoded as a semantic embedding by mapping the concept specifier to a shared embedding space. This modality-independent space can be built by means of deep metric learning using paired data consisting of images and their spoken captions. We use it to bridge modality-dependent information, i.e., the speech segments in the mixture, and the specified, modality-independent concept. As a proof of our scheme, we performed experiments using a set of images associated with spoken captions. That is, we generated speech mixtures from these spoken captions and used the images or speech signals as the concept specifiers. We then extracted the target speech using the acoustic characteristics of the identified segments. We compare ConceptBeam with two methods: one based on keywords obtained from recognition systems and another based on sound source separation. We show that ConceptBeam clearly outperforms the baseline methods and effectively extracts speech based on the semantic representation.
【11】 Learning a Dual-Mode Speech Recognition Model via Self-Pruning
标题： 基于自剪枝的双模式语音识别模型学习
链接：https://arxiv.org/abs/2207.11906
作者：Chunxi Liu,Yuan Shangguan,Haichuan Yang,Yangyang Shi,Raghuraman Krishnamoorthi,Ozlem Kalinli摘要：流媒体语音识别和全上下文自动语音识别的结合越来越引起人们的兴趣（ASR）网络整合到单个端到端ASR模型中，以简化两种使用情形的模型训练和部署。而在实际ASR应用中，流式ASR模型通常在更多存储和计算限制下运行—例如，受Omni-sparsity超网训练的最新进展（其中多个子网在单个模型中联合优化）的启发，本工作旨在联合学习紧凑的稀疏设备上流式ASR模型，和监督ASR微调两者执行超网训练，不仅可以实质上改善如现有工作中所示的大型非流模型，而且还能够改善紧凑稀疏流模型。摘要：There is growing interest in unifying the streaming and full-context automatic speech recognition (ASR) networks into a single end-to-end ASR model to simplify the model training and deployment for both use cases. While in real-world ASR applications, the streaming ASR models typically operate under more storage and computational constraints - e.g., on embedded devices - than any server-side full-context models. Motivated by the recent progress in Omni-sparsity supernet training, where multiple subnetworks are jointly optimized in one single model, this work aims to jointly learn a compact sparse on-device streaming ASR model, and a large dense server non-streaming model, in a single supernet. Next, we present that, performing supernet training on both wav2vec 2.0 self-supervised learning and supervised ASR fine-tuning can not only substantially improve the large non-streaming model as shown in prior works, and also be able to improve the compact sparse streaming model.
【12】 Low-complexity CNNs for Acoustic Scene Classification
标题： 基于低复杂度神经网络的声学场景分类
链接：https://arxiv.org/abs/2207.11529
作者：Arshdeep Singh,Mark D. Plumbley机构：Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK备注：Submitted to DCASE 2022 Workshop摘要：本文提出了一种低复杂度的声学场景分类框架大多数为ASC设计的框架使用卷积神经网络（CNNs）由于其学习能力和与手工设计的特征相比改进的性能。然而，CNNs由于其大的尺寸和高的计算复杂性而需要资源。因此，针对有线网络难以在资源受限的设备上部署的问题，提出了一种低复杂度的有线网络结构，和量化来进一步减少参数和存储量，并提出了一种集成框架，将各种低复杂度的神经网络相结合来提高整体性能。在DCASE2022Task1上对提出的集成框架进行了实验评估，需要19M次乘加运算，与DCASE 2022任务1基线网络相比，性能提高了约2-4个百分点。摘要：This paper presents a low-complexity framework for acoustic scene classification (ASC). Most of the frameworks designed for ASC use convolutional neural networks (CNNs) due to their learning ability and improved performance compared to hand-engineered features. However, CNNs are resource hungry due to their large size and high computational complexity. Therefore, CNNs are difficult to deploy on resource constrained devices. This paper addresses the problem of reducing the computational complexity and memory requirement in CNNs. We propose a low-complexity CNN architecture, and apply pruning and quantization to further reduce the parameters and memory. We then propose an ensemble framework that combines various low-complexity CNNs to improve the overall performance. An experimental evaluation of the proposed framework is performed on the publicly available DCASE 2022 Task 1 that focuses on ASC. The proposed ensemble framework has approximately 60K parameters, requires 19M multiply-accumulate operations and improves the performance by approximately 2-4 percentage points compared to the DCASE 2022 Task 1 baseline network.
【13】 Low-Complexity Acoustic Echo Cancellation with Neural Kalman Filtering
标题： 基于神经卡尔曼滤波的低复杂度声回波对消
链接：https://arxiv.org/abs/2207.11388
作者：Dong Yang,Fei Jiang,Wei Wu,Xuefei Fang,Muyong Cao摘要：卡尔曼滤波器（Kalman Filter）由于其对双端话音的鲁棒性、收敛速度快、稳态性能好等优点，已被广泛应用于声学回波抵消中，Kalman滤波器的性能与状态噪声协方差和观测噪声协方差的估计精度密切相关，估计误差可能导致不可接受的结果，特别是当回波路径发生突变时，本文提出了一种神经卡尔曼滤波方法（NKF）、该方法利用神经网络隐式地对状态噪声和观测噪声的协方差进行建模，并实时地输出Kalman增益，在合成测试集和真实测试集上的实验结果表明，该方法具有很好的鲁棒性和鲁棒性。实验结果表明，与现有的基于模型的方法相比，NKF在保证较低的近端语音质量的同时，具有更好的收敛和再收敛性能，且模型规模仅为5.3K，RTF低至0.09，可以应用于低资源平台.摘要：The Kalman filter has been adopted in acoustic echo cancellation due to its robustness to double-talk, fast convergence, and good steady-state performance. The performance of Kalman filter is closely related to the estimation accuracy of the state noise covariance and the observation noise covariance. The estimation error may lead to unacceptable results, especially when the echo path suffers abrupt changes, the tracking performance of the Kalman filter could be degraded significantly. In this paper, we propose the neural Kalman filtering (NKF), which uses neural networks to implicitly model the covariance of the state noise and observation noise and to output the Kalman gain in real-time. Experimental results on both synthetic test sets and real-recorded test sets show that, the proposed NKF has superior convergence and re-convergence performance while ensuring low near-end speech degradation comparing with the state-of-the-art model-based methods. Moreover, the model size of the proposed NKF is merely 5.3 K and the RTF is as low as 0.09, which indicates that it can be deployed in low-resource platforms.
【1】 Transplantation of Conversational Speaking Style with Interjections in Sequence-to-Sequence Speech Synthesis
标题： 序列到序列语音合成中带感叹词的会话说话风格移植
链接：https://arxiv.org/abs/2207.12262
* 与cs.SD语音【7】为同一篇
作者：Raul Fernandez,David Haws,Guy Lorberbom,Slava Shechtman,Alexander Sorin备注：Accepted for presentation at Interspeech 2022摘要：已知直接从语音序列生成低级声学特征的序列到序列文本到语音体系结构在被提供有足够量的训练数据时产生自然的和有表达力的语音。（在多风格多扬声器设置中），这对于创建可扩展的、可定制的人机交互系统是非常需要的。本文以说话人会话语料库为研究对象，分析了语料库的设计方法，并探讨了在语音转换数据增强的辅助下进行风格迁移的可行性。在一组主观听力实验中，该方法在不降低质量的前提下实现了高保真的风格迁移，但也观察到了一定的语音角色转换，需要进一步改进语音转换。摘要：Sequence-to-Sequence Text-to-Speech architectures that directly generate low level acoustic features from phonetic sequences are known to produce natural and expressive speech when provided with adequate amounts of training data. Such systems can learn and transfer desired speaking styles from one seen speaker to another (in multi-style multi-speaker settings), which is highly desirable for creating scalable and customizable Human-Computer Interaction systems. In this work we explore one-to-many style transfer from a dedicated single-speaker conversational corpus with style nuances and interjections. We elaborate on the corpus design and explore the feasibility of such style transfer when assisted with Voice-Conversion-based data augmentation. In a set of subjective listening experiments, this approach resulted in high-fidelity style transfer with no quality degradation. However, a certain voice persona shift was observed, requiring further improvements in voice conversion.
【2】 Label Uncertainty Modeling and Prediction for Speech Emotion Recognition using t-Distributions
标题： 基于t分布的语音情感识别标签不确定性建模与预测
链接：https://arxiv.org/abs/2207.12135
* 与cs.SD语音【8】为同一篇
作者：Navin Raj Prabhu,Nale Lehmann-Willenbrock,Timo Gerkmann机构：Signal Processing, Universit¨at Hamburg, Hamburg, Germany, Industrial and Organizational Psychology备注：ACCEPTED to ACII 2022 -10th INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING & INTELLIGENT INTERACTION摘要：由于不同的人对其他人的情绪表达的感知不同，因此他们在唤醒和效价方面的注释本身是主观的。为了解决这个问题，这些情绪注释通常由多个注释者收集，并在注释者之间进行平均，以便获得唤醒和效价的标签。然而，除了平均值之外，标签的不确定性也是感兴趣的，并且还应当被建模和预测以用于自动情感识别。在文献中，为了简单起见，通常利用对所收集的注释的高斯假设来进行标签不确定性建模。然而，由于注释器的数量由于资源限制而通常相当小，我们认为高斯方法是一个相当粗糙的假设.相反，在这项工作中，我们建议使用Student's t-分布来建模标签分布，我们推导出相应的基于Kullback-Leibler散度的损失函数并使用它来训练情感标签分布的估计器，通过定性和定量分析，我们在AVEC'16数据集上验证了我们提出的方法。结果表明，我们基于t分布的方法优于高斯方法，具有状态—时间—空间的特性。不确定性建模导致基于语音的情感识别以及最优的甚至更快的收敛。摘要：As different people perceive others' emotional expressions differently, their annotation in terms of arousal and valence are per se subjective. To address this, these emotion annotations are typically collected by multiple annotators and averaged across annotators in order to obtain labels for arousal and valence. However, besides the average, also the uncertainty of a label is of interest, and should also be modeled and predicted for automatic emotion recognition. In the literature, for simplicity, label uncertainty modeling is commonly approached with a Gaussian assumption on the collected annotations. However, as the number of annotators is typically rather small due to resource constraints, we argue that the Gaussian approach is a rather crude assumption. In contrast, in this work we propose to model the label distribution using a Student's t-distribution which allows us to account for the number of annotations available. With this model, we derive the corresponding Kullback-Leibler divergence based loss function and use it to train an estimator for the distribution of emotion labels, from which the mean and uncertainty can be inferred. Through qualitative and quantitative analysis, we show the benefits of the t-distribution over a Gaussian distribution. We validate our proposed method on the AVEC'16 dataset. Results reveal that our t-distribution based approach improves over the Gaussian approach with state-of-the-art uncertainty modeling results in speech-based emotion recognition, along with an optimal and even faster convergence.
【3】 A Polyphone BERT for Polyphone Disambiguation in Mandarin Chinese
标题： 汉语多音字消歧的多音字BERT算法
链接：https://arxiv.org/abs/2207.12089
作者：Song Zhang,Ken Zheng,Xiaoxu Zhu,Baoxiang Li备注：Accepted for INTERSPEECH 2022摘要：字形转音素（G2P）转换是汉语普通话文语转换中不可缺少的一部分（TTS）系统中，G2P转换的核心是解决多音字的消歧问题，即对一个多音字的多个候选字进行正确的发音，本文提出了一个多音字的BERT模型，该算法首先从354个源多音字中按发音生成741个新的汉语单音字，然后用741个新的汉语单音字扩展一个预先训练好的汉语单音字BERT，并在BERT中增加一个新的标记嵌入层，将多音字消歧任务转化为汉语多音字BERT的预训练任务，实验结果表明了该模型的有效性，多音字BERT模型取得了2%的正确率.与基于BERT的分类器模型（其是多音字消歧中的现有技术）相比，平均准确度的改进（从92.1%到94.1%）。摘要：Grapheme-to-phoneme (G2P) conversion is an indispensable part of the Chinese Mandarin text-to-speech (TTS) system, and the core of G2P conversion is to solve the problem of polyphone disambiguation, which is to pick up the correct pronunciation for several candidates for a Chinese polyphonic character. In this paper, we propose a Chinese polyphone BERT model to predict the pronunciations of Chinese polyphonic characters. Firstly, we create 741 new Chinese monophonic characters from 354 source Chinese polyphonic characters by pronunciation. Then we get a Chinese polyphone BERT by extending a pre-trained Chinese BERT with 741 new Chinese monophonic characters and adding a corresponding embedding layer for new tokens, which is initialized by the embeddings of source Chinese polyphonic characters. In this way, we can turn the polyphone disambiguation task into a pre-training task of the Chinese polyphone BERT. Experimental results demonstrate the effectiveness of the proposed model, and the polyphone BERT model obtain 2% (from 92.1% to 94.1%) improvement of average accuracy compared with the BERT-based classifier model, which is the prior state-of-the-art in polyphone disambiguation.
【4】 Unsupervised data selection for Speech Recognition with contrastive loss ratios
标题： 基于对比丢失率的语音识别无监督数据选择
链接：https://arxiv.org/abs/2207.12028
* 与cs.SD语音【9】为同一篇
作者：Chanho Park,Rehan Ahmad,Thomas Hain机构：Department of Computer Science, University of Sheffield摘要：本文提出了一种基于目标数据集和训练数据集的对比丢失率的子模函数无监督数据选择方法.首先在目标数据集和训练数据集上训练一个使用对比丢失函数的模型.然后，实验结果表明，用该方法选择的训练集训练的模型性能优于基于对数—方差模型的选择方法.由GMM-HMM模型产生的似然性，以单词错误率表示（WER）。当选择固定数量（例如10小时的数据）时，两种方法在Teddalk上的结果之间的差异为20.23%相对WER。该方法也可用于选择数据，目的是使负迁移最小化，同时保持或提高了在整个训练集上训练的模型的性能，结果表明，在WSJCAM0数据集上，当从整个数据集中选取85%时，WER相对降低了6.26%。摘要：This paper proposes an unsupervised data selection method by using a submodular function based on contrastive loss ratios of target and training data sets. A model using a contrastive loss function is trained on both sets. Then the ratio of frame-level losses for each model is used by a submodular function. By using the submodular function, a training set for automatic speech recognition matching the target data set is selected. Experiments show that models trained on the data sets selected by the proposed method outperform the selection method based on log-likelihoods produced by GMM-HMM models, in terms of word error rate (WER). When selecting a fixed amount, e.g. 10 hours of data, the difference between the results of two methods on Tedtalks was 20.23% WER relative. The method can also be used to select data with the aim of minimising negative transfer, while maintaining or improving on performance of models trained on the whole training set. Results show that the WER on the WSJCAM0 data set was reduced by 6.26% relative when selecting 85% from the whole data set.
【5】 ConceptBeam: Concept Driven Target Speech Extraction
标题： 概念梁：概念驱动的目标语音提取
链接：https://arxiv.org/abs/2207.11964
* 与cs.SD语音【10】为同一篇
作者：Yasunori Ohishi,Marc Delcroix,Tsubasa Ochiai,Shoko Araki,Daiki Takeuchi,Daisuke Niizumi,Akisato Kimura,Noboru Harada,Kunio Kashino机构：NTT Corporation, Tokyo, Japan备注：Accepted to ACM Multimedia 2022摘要：提出了一种基于语义信息的目标语音提取框架ConceptBeam.目标语音提取是指从混合语音中提取目标说话人的语音.典型的方法是利用音频信号的特性，如谐波结构和到达方向.相比之下，ConceptBeam则利用语义线索来处理这个问题.具体来说，我们提取说话人谈论一个概念的语音，即感兴趣的主题。解决这个新颖的问题将打开通向创新应用的大门，例如专注于在对话中讨论的特定主题的收听系统。与关键字不同，概念是抽象概念，使得直接表示目标概念具有挑战性。在我们的方案中，通过将概念描述符映射到一个共享嵌入空间，将概念编码为语义嵌入，该嵌入空间可以通过深度度量学习的方式使用由图像和其语音字幕组成的成对数据来建立，该方法利用了一组与语音字幕相关联的图像来进行实验，即，我们从这些字幕中产生语音混合，并使用图像或语音信号作为概念描述符，然后我们利用识别片段的声学特征提取目标语音。我们将ConceptBeam与两种方法进行比较：一个基于从识别系统获得的关键词，另一个基于声源分离，我们表明ConceptBeam明显优于基线方法，并有效地提取基于语义表示的语音。摘要：We propose a novel framework for target speech extraction based on semantic information, called ConceptBeam. Target speech extraction means extracting the speech of a target speaker in a mixture. Typical approaches have been exploiting properties of audio signals, such as harmonic structure and direction of arrival. In contrast, ConceptBeam tackles the problem with semantic clues. Specifically, we extract the speech of speakers speaking about a concept, i.e., a topic of interest, using a concept specifier such as an image or speech. Solving this novel problem would open the door to innovative applications such as listening systems that focus on a particular topic discussed in a conversation. Unlike keywords, concepts are abstract notions, making it challenging to directly represent a target concept. In our scheme, a concept is encoded as a semantic embedding by mapping the concept specifier to a shared embedding space. This modality-independent space can be built by means of deep metric learning using paired data consisting of images and their spoken captions. We use it to bridge modality-dependent information, i.e., the speech segments in the mixture, and the specified, modality-independent concept. As a proof of our scheme, we performed experiments using a set of images associated with spoken captions. That is, we generated speech mixtures from these spoken captions and used the images or speech signals as the concept specifiers. We then extracted the target speech using the acoustic characteristics of the identified segments. We compare ConceptBeam with two methods: one based on keywords obtained from recognition systems and another based on sound source separation. We show that ConceptBeam clearly outperforms the baseline methods and effectively extracts speech based on the semantic representation.
【6】 Learning a Dual-Mode Speech Recognition Model via Self-Pruning
标题： 基于自剪枝的双模式语音识别模型学习
链接：https://arxiv.org/abs/2207.11906
* 与cs.SD语音【11】为同一篇
作者：Chunxi Liu,Yuan Shangguan,Haichuan Yang,Yangyang Shi,Raghuraman Krishnamoorthi,Ozlem Kalinli摘要：流媒体语音识别和全上下文自动语音识别的结合越来越引起人们的兴趣（ASR）网络整合到单个端到端ASR模型中，以简化两种使用情形的模型训练和部署。而在实际ASR应用中，流式ASR模型通常在更多存储和计算限制下运行—例如，受Omni-sparsity超网训练的最新进展（其中多个子网在单个模型中联合优化）的启发，本工作旨在联合学习紧凑的稀疏设备上流式ASR模型，和监督ASR微调两者执行超网训练，不仅可以实质上改善如现有工作中所示的大型非流模型，而且还能够改善紧凑稀疏流模型。摘要：There is growing interest in unifying the streaming and full-context automatic speech recognition (ASR) networks into a single end-to-end ASR model to simplify the model training and deployment for both use cases. While in real-world ASR applications, the streaming ASR models typically operate under more storage and computational constraints - e.g., on embedded devices - than any server-side full-context models. Motivated by the recent progress in Omni-sparsity supernet training, where multiple subnetworks are jointly optimized in one single model, this work aims to jointly learn a compact sparse on-device streaming ASR model, and a large dense server non-streaming model, in a single supernet. Next, we present that, performing supernet training on both wav2vec 2.0 self-supervised learning and supervised ASR fine-tuning can not only substantially improve the large non-streaming model as shown in prior works, and also be able to improve the compact sparse streaming model.
【7】 Low-complexity CNNs for Acoustic Scene Classification
标题： 基于低复杂度神经网络的声学场景分类
链接：https://arxiv.org/abs/2207.11529
* 与cs.SD语音【12】为同一篇
作者：Arshdeep Singh,Mark D. Plumbley机构：Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK备注：Submitted to DCASE 2022 Workshop摘要：本文提出了一种低复杂度的声学场景分类框架大多数为ASC设计的框架使用卷积神经网络（CNNs）由于其学习能力和与手工设计的特征相比改进的性能。然而，CNNs由于其大的尺寸和高的计算复杂性而需要资源。因此，针对有线网络难以在资源受限的设备上部署的问题，提出了一种低复杂度的有线网络结构，和量化来进一步减少参数和存储量，并提出了一种集成框架，将各种低复杂度的神经网络相结合来提高整体性能。在DCASE2022Task1上对提出的集成框架进行了实验评估，需要19M次乘加运算，与DCASE 2022任务1基线网络相比，性能提高了约2-4个百分点。摘要：This paper presents a low-complexity framework for acoustic scene classification (ASC). Most of the frameworks designed for ASC use convolutional neural networks (CNNs) due to their learning ability and improved performance compared to hand-engineered features. However, CNNs are resource hungry due to their large size and high computational complexity. Therefore, CNNs are difficult to deploy on resource constrained devices. This paper addresses the problem of reducing the computational complexity and memory requirement in CNNs. We propose a low-complexity CNN architecture, and apply pruning and quantization to further reduce the parameters and memory. We then propose an ensemble framework that combines various low-complexity CNNs to improve the overall performance. An experimental evaluation of the proposed framework is performed on the publicly available DCASE 2022 Task 1 that focuses on ASC. The proposed ensemble framework has approximately 60K parameters, requires 19M multiply-accumulate operations and improves the performance by approximately 2-4 percentage points compared to the DCASE 2022 Task 1 baseline network.
【8】 Low-Complexity Acoustic Echo Cancellation with Neural Kalman Filtering
标题： 基于神经卡尔曼滤波的低复杂度声回波对消
链接：https://arxiv.org/abs/2207.11388
* 与cs.SD语音【13】为同一篇
作者：Dong Yang,Fei Jiang,Wei Wu,Xuefei Fang,Muyong Cao摘要：卡尔曼滤波器（Kalman Filter）由于其对双端话音的鲁棒性、收敛速度快、稳态性能好等优点，已被广泛应用于声学回波抵消中，Kalman滤波器的性能与状态噪声协方差和观测噪声协方差的估计精度密切相关，估计误差可能导致不可接受的结果，特别是当回波路径发生突变时，本文提出了一种神经卡尔曼滤波方法（NKF）、该方法利用神经网络隐式地对状态噪声和观测噪声的协方差进行建模，并实时地输出Kalman增益，在合成测试集和真实测试集上的实验结果表明，该方法具有很好的鲁棒性和鲁棒性。实验结果表明，与现有的基于模型的方法相比，NKF在保证较低的近端语音质量的同时，具有更好的收敛和再收敛性能，且模型规模仅为5.3K，RTF低至0.09，可以应用于低资源平台.摘要：The Kalman filter has been adopted in acoustic echo cancellation due to its robustness to double-talk, fast convergence, and good steady-state performance. The performance of Kalman filter is closely related to the estimation accuracy of the state noise covariance and the observation noise covariance. The estimation error may lead to unacceptable results, especially when the echo path suffers abrupt changes, the tracking performance of the Kalman filter could be degraded significantly. In this paper, we propose the neural Kalman filtering (NKF), which uses neural networks to implicitly model the covariance of the state noise and observation noise and to output the Kalman gain in real-time. Experimental results on both synthetic test sets and real-recorded test sets show that, the proposed NKF has superior convergence and re-convergence performance while ensuring low near-end speech degradation comparing with the state-of-the-art model-based methods. Moreover, the model size of the proposed NKF is merely 5.3 K and the RTF is as low as 0.09, which indicates that it can be deployed in low-resource platforms.
【9】 FAD: A Chinese Dataset for Fake Audio Detection
标题： FAD：一个用于虚假音频检测的中文数据集
链接：https://arxiv.org/abs/2207.12308
* 与cs.SD语音【11】为同一篇
作者：Haoxin Ma,Jiangyan Yi,Chenglong Wang,Xinrui Yan,Jianhua Tao,Tao Wang,Shiming Wang,Le Xu,Ruibo Fu机构：NLPR, Institute of Automation, Chinese Academy of Sciences, China, School of Artificial Intelligence, University of Chinese Academy of Sciences, China, CAS Center for Excellence in Brain Science and Intelligence Technology, China备注：submitted to NeurIPS 2022 Track on Datasets and Benchmarks摘要：虚假音频检测是一个日益受到关注的问题，已经设计了一些相关的数据集用于研究，但是在加性噪声条件下，还没有一个标准的公开的中文数据集，本文旨在填补这一空白，设计一个中文虚假音频检测数据集（FAD）来研究更一般的检测方法.使用12种主流的语音产生技术来产生假音频.为了模拟真实的场景，在5种不同的信噪比下，选择3个噪声数据集进行加噪. FAD数据集不仅可以用于虚假音频检测，而且可以用于音频取证中的虚假话语检测算法.给出了基线结果并进行了分析.结果表明，具有泛化能力的虚假音频检测方法仍然具有挑战性. FAD数据集是公开的.摘要：Fake audio detection is a growing concern and some relevant datasets have been designed for research. But there is no standard public Chinese dataset under additive noise conditions. In this paper, we aim to fill in the gap and design a Chinese fake audio detection dataset (FAD) for studying more generalized detection methods. Twelve mainstream speech generation techniques are used to generate fake audios. To simulate the real-life scenarios, three noise datasets are selected for noisy adding at five different signal noise ratios. FAD dataset can be used not only for fake audio detection, but also for detecting the algorithms of fake utterances for audio forensics. Baseline results are presented with analysis. The results that show fake audio detection methods with generalization remain challenging. The FAD dataset is publicly available.
【10】 Domain Adapting Speech Emotion Recognition modals to real-world scenario with Deep Reinforcement Learning
标题： 基于深度强化学习的领域自适应语音情感识别模型
链接：https://arxiv.org/abs/2207.12248
* 与cs.SD语音【2】为同一篇
作者：Thejan Rajapakshe,Rajib Rana,Sara Khalifa机构： Rajapakshe is affiliated with University of Southern Queensland, Rana is with University of Southern Queensland摘要：随着深度学习在机器学习领域的广泛应用，深度强化学习已经成为一种流行的训练范式.领域自适应允许我们在训练一个阶段后跨领域转移由模型学习到的知识.现有领域自适应算法的缺点之一是不能使现有模型适应真实世界的领域.本文提出了一种深度强化学习算法———————————-本文提出了一种基于自适应策略的语音情感识别方法，该方法可以在与环境交互的同时，不断地收集反馈信息，使预先训练好的模型适应新的领域，并将该方法应用于语音情感识别任务中，我们的方法在跨语料库和跨语言模式上分别比监督学习策略的性能高出42%和20%。摘要：Deep reinforcement learning has been a popular training paradigm as deep learning has gained popularity in the field of machine learning. Domain adaptation allows us to transfer knowledge learnt by a model across domains after a phase of training. The inability to adapt an existing model to a real-world domain is one of the shortcomings of current domain adaptation algorithms. We present a deep reinforcement learning-based strategy for adapting a pre-trained model to a newer domain while interacting with the environment and collecting continual feedback. This method was used on the Speech Emotion Recognition task, which included both cross-corpus and cross-language domain adaption schema. Furthermore, it demonstrates that in a real-world environment, our approach outperforms the supervised learning strategy by 42% and 20% in cross-corpus and cross-language schema, respectively.
【11】 Cross-Modal Contrastive Representation Learning for Audio-to-Image Generation
标题： 基于跨通道对比表示学习的音像转换算法
链接：https://arxiv.org/abs/2207.12121
* 与cs.SD语音【3】为同一篇
作者：HaeChun Chung,JooYong Shim,Jong-Kook Kim机构：Dept. of Electrical Engineering, Korea University备注：7 pages, 3 figures, Accepted to MUE 2022摘要：多模态信息提供了对信息的多种视角，可以提高对信息的理解，因此，从已有的数据中生成不同模态的数据对于提高理解可能是至关重要的.本文提出了一种基于多模态数据的信息检索方法，我们研究了跨通道音频到图像的生成问题，并提出了跨通道对比表征学习实验结果表明，CMCRL算法能够从音频中提取有用的特征，并将其应用于图像生成阶段，提高了生成图像的质量。摘要：Multiple modalities for certain information provide a variety of perspectives on that information, which can improve the understanding of the information. Thus, it may be crucial to generate data of different modality from the existing data to enhance the understanding. In this paper, we investigate the cross-modal audio-to-image generation problem and propose Cross-Modal Contrastive Representation Learning (CMCRL) to extract useful features from audios and use it in the generation phase. Experimental results show that CMCRL enhances quality of images generated than previous research.
【12】 Source Separation of Unknown Numbers of Single-Channel Underwater Acoustic Signals Based on Autoencoders
标题： 基于自编码器的未知数目单通道水声信号源分离
链接：https://arxiv.org/abs/2207.11749
* 与cs.SD语音【4】为同一篇
作者：Qinggang Sun,Kejun Wang机构：Q. Sun and K. Wang are with the College of Intelligent Systems Science and, Engineering, Harbin Engineering University, Harbin, Heilongjiang, China, and, K. Wang is also with the School of Information Technology, Beijing Institute, Zhuhai, Guangdong, (e-mails:备注：6 pages, 2 figures,3 tables. For codes, see this https URL This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible摘要：单通道水声信号的分离是一个具有实际意义的难题，针对未知个数的信号分离问题，提出了一种输出通道数固定的信号分离方法，避免了由于输出与目标对准而引起的排列问题所带来的维数灾难，具体而言，我们修改了基于自动编码器为已知数量的信号开发的两种算法，并提出了一种新的静音信道下的性能评估方法。在模拟的舰船辐射噪声混合信号上的实验表明，该方法可以获得与已知信号数时相似的分离性能。静音通道输出也不错。摘要：The separation of single-channel underwater acoustic signals is a challenging problem with practical significance. In view of the signal separation problem with unknown numbers of signals, we propose a solution with a fixed number of output channels, enabling it to avoid the dimensional disaster caused by the permutation problem induced by the alignment of outputs to targets. Specifically, we modify two algorithms developed for known numbers of signals based on autoencoders, which are highly explainable. We also propose a new performance evaluation method for situations with mute channels. Experiments conducted on simulated mixtures of radiated ship noise show that the proposed solution can achieve similar separation performance to that attained with a known number of signals. The mute channel output is also good.
【13】 HouseX: A Fine-grained House Music Dataset and its Potential in the Music Industry
标题： HouseX：一个细粒度的House音乐数据集及其在音乐产业中的潜力
链接：https://arxiv.org/abs/2207.11690
* 与cs.SD语音【5】为同一篇
机构：∗ New York University Shanghai, Shanghai, China摘要：机器声音分类一直是音乐技术的基本任务之一.声音分类的一个主要分支是音乐流派的分类.然而，尽管现有的音乐流派数据集涵盖了大多数音乐流派，但往往不包含细粒度的标签来指示音乐的详细子流派.考虑到混音带或DJ中歌曲流派的一致性，提出了一种基于机器声音分类的音乐流派分类方法.该方法基于机器声音分类的基本思想，利用机器声音分类的基本思想，提出了一种基于机器声音分类的音乐流派分类方法.该方法基于机器声音分类的基本思想，提出了一种基于机器声音分类的音乐流派分类方法本文在现场音乐背景下，收集了一个家庭音乐数据集，并对该数据集进行了标注，给出了未来家庭音乐、低音家庭音乐、渐进家庭音乐和旋律家庭音乐4个子类型的标注.实验表明，我们的标注很好地体现了不同类型的特点.同时，我们还建立了基于旋律谱图的子类型分类基线模型，此外，我们还提出了我们的数据集和基准模型的几个应用场景，其中一个模拟的科幻隧道作为一个简短的演示，在3D建模软件中构建和渲染，灯光的颜色由我们的模型的输出自动化。摘要：Machine sound classification has been one of the fundamental tasks of music technology. A major branch of sound classification is the classification of music genres. However, though covering most genres of music, existing music genre datasets often do not contain fine-grained labels that indicate the detailed sub-genres of music. In consideration of the consistency of genres of songs in a mixtape or in a DJ (live) set, we have collected and annotated a dataset of house music that provide 4 sub-genre labels, namely future house, bass house, progressive house and melodic house. Experiments show that our annotations well exhibit the characteristics of different categories. Also, we have built baseline models that classify the sub-genre based on the mel-spectrograms of a track, achieving strongly competitive results. Besides, we have put forward a few application scenarios of our dataset and baseline model, with a simulated sci-fi tunnel as a short demo built and rendered in a 3D modeling software, with the colors of the lights automated by the output of our model.
【14】 Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparities
标题： 语音识别中的公平性：发现和缓解性能差异
链接：https://arxiv.org/abs/2207.11345
* 与cs.SD语音【6】为同一篇
作者：Pranav Dheram,Murugesan Ramakrishnan,Anirudh Raju,I-Fan Chen,Brian King,Katherine Powell,Melissa Saboowala,Karan Shetty,Andreas Stolcke备注：Proc. Interspeech 2022摘要：至于其他形式的人工智能，最近已经就不同用户群体之间的性能差异对语音识别进行了研究。（1）识别遭受低于标准表现的说话者群组，以及（2）针对发现的群组应用公平性缓解措施，我们报告了使用来自产品级AI辅助语音识别系统的数据发现和减轻性能差异的初步发现。我们将基于地理和人口统计信息的群组发现与使用说话人嵌入技术的更可扩展的无人类标签的说话人分组方法进行比较。我们发现，对代表不足的群组进行过采样以及通过附加输入变量对说话者群组成员资格进行建模，减小了表现最好和最差的群组之间的差距，而不会使整体识别准确度恶化。摘要：As for other forms of AI, speech recognition has recently been examined with respect to performance disparities across different user cohorts. One approach to achieve fairness in speech recognition is to (1) identify speaker cohorts that suffer from subpar performance and (2) apply fairness mitigation measures targeting the cohorts discovered. In this paper, we report on initial findings with both discovery and mitigation of performance disparities using data from a product-scale AI assistant speech recognition system. We compare cohort discovery based on geographic and demographic information to a more scalable method that groups speakers without human labels, using speaker embedding technology. For fairness mitigation, we find that oversampling of underrepresented cohorts, as well as modeling speaker cohort membership by additional input variables, reduces the gap between top- and bottom-performing cohorts, without deteriorating overall recognition accuracy.
机器翻译，仅供参考
每日语音速递