语音/音频处理学术速递[2.3]

今日论文合集：cs.SD语音4篇，eess.AS音频处理4篇。

cs.SD语音

【1】 Goniometers are a Powerful Acoustic Feature for Music Information Retrieval Tasks

标题：测角器是音乐信息检索任务的强大声学功能

链接：https://arxiv.org/abs/2302.01090

作者：Tim Ziemer

机构：Goniometers are a Powerful Acoustic Feature for Music Information Retrieval TasksTim Ziemer 1 1 University of Hamburg

摘要：测角仪，也称为相位示波器或矢量示波器，是音频测量工具，帮助音乐制作人和混音工程师监测音乐混音的空间方面，如立体声全景，单一来源的宽度，混响的数量和扩散，以及相位抵消，可能发生在甜蜜点和单声道混音。此外，它们还隐含地告知声音的动态。使用角度计训练的自组织映射，被用来探索这种声学特征在音乐信息检索任务中的有用性。人们可以看到，角度计能够分类不同的流派和集群一个单一的专辑。角度计的优势在于因果关系：音乐制作人和混音工程师有意识地使用测角器来获得想要的声音，而其他声学特性（从过零率到梅尔频率倒谱系数）则不是这样。

摘要：Goniometers, also known as Phase Scopes or Vector Scopes, are audio metering tools that help music producers and mixing engineers monitor spatial aspects of a music mix, such as the stereo panorama, the width of single sources, the amount and diffuseness of reverberation as well as phase cancellations that may occur on the sweet-spot and in a mono-mixdown. In addition, they implicitly inform about the dynamics of the sound. Self-organizing maps trained with a goniometer, are consulted to explore the usefulness of this acoustic feature for music information retrieval tasks. One can see that goniometers are able to classify different genres and cluster a single album. The advantage of goniometers is the causality: Music producers and mixing engineers consciously consult goniometers to reach their desired sound, which is not the case for other acoustic features, from Zero-Crossing Rate to Mel-Frequency Cepstral Coefficients.

【2】 Speech Enhancement for Virtual Meetings on Cellular Networks

标题：蜂窝网络虚拟会议的语音增强

链接：https://arxiv.org/abs/2302.00868

作者：Hojeong Lee,Minseon Gwak,Kawon Lee,Minjeong Kim,Joseph Konan,Ojas Bhargave

机构：Carnegie Mellon University, Pittsburgh, PA

摘要：我们研究了使用深度学习（DL）在蜂窝设备上进行虚拟会议的语音增强，其中传输的语音具有影响语音质量的背景噪声和传输损耗。由于深度噪声抑制（DNS）挑战数据集不包含实际干扰，我们使用T-Mobile网络上的Zoom Meetings收集传输的DNS（t-DNS）数据集。我们选择两个基线模型：Demucs和完整子网。Demucs是端到端模型，其获取时域输入并输出时域降噪语音，而FullSubNet获取时频域输入并输出目标语音在输入中的能量比。该项目的目标是使用深度学习模型增强通过蜂窝网络传输的语音。

摘要：We study speech enhancement using deep learning (DL) for virtual meetings on cellular devices, where transmitted speech has background noise and transmission loss that affects speech quality. Since the Deep Noise Suppression (DNS) Challenge dataset does not contain practical disturbance, we collect a transmitted DNS (t-DNS) dataset using Zoom Meetings over T-Mobile network. We select two baseline models: Demucs and FullSubNet. The Demucs is an end-to-end model that takes time-domain inputs and outputs time-domain denoised speech, and the FullSubNet takes time-frequency-domain inputs and outputs the energy ratio of the target speech in the inputs. The goal of this project is to enhance the speech transmitted over the cellular networks using deep learning models.

【3】 Improving Rare Words Recognition through Homophone Extension and Unified Writing for Low-resource Cantonese Speech Recognition

标题：在低资源粤语语音识别中通过同音扩展和统一书写改进生词识别

链接：https://arxiv.org/abs/2302.00836

作者：HoLam Chung,Junan Li,Pengfei Liu1,Wai-Kim Leung,Xixin Wu,Helen Meng

机构：Centre for Perceptual and Interactive Intelligence (CPII), Hong Kong, The Chinese University of Hong Kong, Hong Kong, Graduate Institute of Electrical Engineering, National Taiwan University, Taiwan备注：None

摘要：同音字在基于声调音节的语言中是常见的，例如普通话和广东话。数据密集型的端到端自动语音识别系统在低资源环境下更容易误识别同音字和生僻字。针对广东话语音识别中资源匮乏的问题，提出了一种新的同音词扩展方法，将人类对同音词词典的知识融入到语音模型重评分的波束搜索解码过程中。此外，我们提出了一种自动化的统一书写方法来合并粤语字的变体并规范语音标注准则，通过为合并后的字提供更多的样本来更有效地利用标注的话语。实验结果表明，同音字扩展和统一书写在域内和域外测试集上都显著提高了识别性能，绝对字符错误率（CER）分别降低了约5%和18%。

摘要：Homophone characters are common in tonal syllable-based languages, such as Mandarin and Cantonese. The data-intensive end-to-end Automatic Speech Recognition (ASR) systems are more likely to mis-recognize homophone characters and rare words under low-resource settings. For the problem of lowresource Cantonese speech recognition, this paper presents a novel homophone extension method to integrate human knowledge of the homophone lexicon into the beam search decoding process with language model re-scoring. Besides, we propose an automatic unified writing method to merge the variants of Cantonese characters and standardize speech annotation guidelines, which enables more efficient utilization of labeled utterances by providing more samples for the merged characters. We empirically show that both homophone extension and unified writing improve the recognition performance significantly on both in-domain and out-of-domain test sets, with an absolute Character Error Rate (CER) decrease of around 5% and 18%.

【4】 Visually Grounded Keyword Detection and Localisation for Low-Resource Languages

标题：基于视觉的低资源语言关键词检测与本地化

链接：https://arxiv.org/abs/2302.00765

作者：Kayode Kolawole Olaleye

机构：Dissertation presented for the degree of Doctor of Philosophy (Electronic, Supervisor: Prof. Herman Kamper, arXiv:,.,v, [cs.CL] , Feb

备注：PhD dissertation, University of Stellenbosch, 108 pages, submitted and accepted 2023

摘要：本研究探讨视觉扎根语音（VGS）模型在关键词定位中的应用。本研究主要关注两个研究问题：（1）关键字本地化是否可能与VGS模型和（2）关键字本地化是否可以跨语言在一个真正的低资源设置？提出了四种定位方法，并在英文数据集上进行了评估，其中性能最好的方法达到了57%的准确度。还收集并发布了一个新的数据集，其中包含约鲁巴语的口头字幕，用于跨语言关键词本地化。跨语言模型在实际的关键词本地化中获得16%的精确度，并且该性能可以通过从在英语数据上预训练的模型初始化来提高。该研究详细分析了该模型的成功和失败模式，并强调了在低资源环境下使用VGS模型进行关键词本地化的挑战。

摘要：This study investigates the use of Visually Grounded Speech (VGS) models for keyword localisation in speech. The study focusses on two main research questions: (1) Is keyword localisation possible with VGS models and (2) Can keyword localisation be done cross-lingually in a real low-resource setting? Four methods for localisation are proposed and evaluated on an English dataset, with the best-performing method achieving an accuracy of 57%. A new dataset containing spoken captions in Yoruba language is also collected and released for cross-lingual keyword localisation. The cross-lingual model obtains a precision of 16% in actual keyword localisation and this performance can be improved by initialising from a model pretrained on English data. The study presents a detailed analysis of the model's success and failure modes and highlights the challenges of using VGS models for keyword localisation in low-resource settings.

eess.AS音频处理