今日论文合集：cs.SD语音6篇，eess.AS音频处理6篇。
cs.SD语音
【1】 The ACM Multimedia 2023 Computational Paralinguistics Challenge: Emotion  Share & Requests标题：ACM多媒体2023计算副语言学挑战赛：情感分享与请求链接：https://arxiv.org/abs/2304.14882
作者：Björn W. Schuller,Anton Batliner,Shahin Amiriparian,Alexander Barnhill,Maurice Gerczuk,Andreas Triantafyllopoulos,Alice Baird,Panagiotis Tzirakis,Chris Gagne,Alan S. Cowen,Nikola Lackovic,Marie-José Caraty机构：GLAM, Imperial College London, London, United Kingdom, EIHW, University of Augsburg, Augsburg, Germany, FAU, Erlangen-Nuremberg, Germany, Hume AI, New York, USA, Malakoff Humanis, Paris, France, Sorbonne University, Claude Montacié备注：5 pages, part of the ACM Multimedia 2023 Grand Challenge "The ACM Multimedia 2023 Computational Paralinguistics Challenge (ComParE 2023). arXiv admin note: text overlap with arXiv:2205.06799摘要：ACM Multimedia 2023 Computational Paralinguistics Challenge首次在明确定义的条件下在研究竞赛中解决了两个不同的问题：在情感分享子挑战中，必须对言语进行回归;在请求子质询中，需要检测请求和投诉。我们描述了子挑战、基线特征提取和基于通常的CompPaRE特征的分类器、auDeep工具包以及使用DeepSpectRum工具包从预训练的CNN中提取深度特征;此外，使用wav 2 vec 2模型。摘要：The ACM Multimedia 2023 Computational Paralinguistics Challenge addresses two different problems for the first time in a research competition under well-defined conditions: In the Emotion Share Sub-Challenge, a regression on speech has to be made; and in the Requests Sub-Challenges, requests and complaints need to be detected. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the usual ComPaRE features, the auDeep toolkit, and deep feature extraction from pre-trained CNNs using the DeepSpectRum toolkit; in addition, wav2vec2 models are used.
【2】 Musical Voice Separation as Link Prediction: Modeling a Musical  Perception Task as a Multi-Trajectory Tracking Problem标题：作为链接预测的音乐声音分离：将音乐感知任务建模为多轨迹跟踪问题链接：https://arxiv.org/abs/2304.14848
作者：Emmanouil Karystinaios,Francesco Foscarin,Gerhard Widmer机构：Johannes Kepler University Linz, Austria, LIT AI Lab, Linz Institute of Technology, Austria, † Equal contribution among authors.备注：Accepted at the 32nd International Joint Conference on Artificial Intelligence (IJCAI-23)摘要：本文的目标是分离不同的交互声音的知觉任务，即。例如，在一个复调音乐作品中的单声道旋律流。我们的目标是符号音乐，音符被明确编码，并将此任务建模为离散观测的多轨迹跟踪（MTT）问题。例如，在一个音高时间空间中的音符。我们的方法从一个音乐作品，通过创建一个节点为每个音符，并通过预测两个音符之间的链接，如果他们是连续的在同一个声音 流分离的旋律轨迹。这种局部的贪婪预测是可能的节点嵌入创建的异构图神经网络，可以捕获内部和内部轨迹信息。此外，我们提出了一个新的正则化损失，鼓励输出尊重MTT的前提下，最多一个传入和一个传出链路的每个节点，有利于单声道（语音）轨迹;该损失函数在其它一般MTT情形中也可能是有用的。我们的方法不使用特定领域的启发式，可扩展到更长的序列和更高数量的声音，并可以处理复杂的情况下，如语音反转和重叠。我们达到新的国家的最先进的结果，在不同风格的古典音乐的声音分离任务。摘要：This paper targets the perceptual task of separating the different interacting voices, i.e., monophonic melodic streams, in a polyphonic musical piece. We target symbolic music, where notes are explicitly encoded, and model this task as a Multi-Trajectory Tracking (MTT) problem from discrete observations, i.e., notes in a pitch-time space. Our approach builds a graph from a musical piece, by creating one node for every note, and separates the melodic trajectories by predicting a link between two notes if they are consecutive in the same voice stream. This kind of local, greedy prediction is made possible by node embeddings created by a heterogeneous graph neural network that can capture inter- and intra-trajectory information. Furthermore, we propose a new regularization loss that encourages the output to respect the MTT premise of at most one incoming and one outgoing link for every node, favouring monophonic (voice) trajectories; this loss function might also be useful in other general MTT scenarios. Our approach does not use domain-specific heuristics, is scalable to longer sequences and a higher number of voices, and can handle complex cases such as voice inversions and overlaps. We reach new state-of-the-art results for the voice separation task in classical music of different styles.
【3】 A noise-robust acoustic method for recognition of foraging activities of  grazing cattle标题：一种识别放牧牛觅食行为的抗噪声学方法链接：https://arxiv.org/abs/2304.14824
作者：Luciano S. Martinez-Rau,José O. Chelotti,Mariano Ferrero,Julio R. Galli,Santiago A. Utsumi,Alejandra M. Planisich,H. Leonardo Rufiner,Leonardo L. Giovanini机构：Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNLCONICET, Argentina, Department of Computer and Electrical Engineering, Mid Sweden University Sundsvall备注：list of used audio-clips is available in the list_audio_clips.xlsx摘要：为了在不断增长的乳制品市场中保持竞争力，农民必须不断改进他们的畜牧生产系统。精准畜牧技术为商业农场提供个性化的动物监测，优化畜牧生产。连续声学监测是一种被广泛接受的传感技术，用于估计自由放养牛的每日反刍和放牧时间预算。然而，牧场上典型的环境和自然噪声显著影响当前声学方法的性能和泛化。在这项研究中，我们提出了一种声学方法称为噪声鲁棒觅食活动识别器（NRFAR）。所提出的方法确定觅食活动回合通过分析固定长度段确定的下颌运动事件与放牧和反刍。使用平稳高斯白噪声和四种不同的非平稳自然噪声源，NRFAR的加性噪声鲁棒性进行了评估的几个信号-噪声比。在无噪声条件下，NRFAR达到了89%的平均平衡精度，比以前的两种声学方法高出7%以上。此外，NRFAR在80个评估的噪声场景中的66个中呈现出比先前声学方法更好的性能（p〈0.01）。NRFAR在线操作，具有与先前声学方法类似的计算成本。这些特性的组合和在恶劣的自由测距环境中的高性能使NRFAR成为在低功耗嵌入式设备中实时实现的绝佳选择。本出版物中呈现的仪器和计算算法受未决专利申请的保护：AR P20220100910。Web演示网址：www.example.com摘要：To stay competitive in the growing dairy market, farmers must continuously improve their livestock production systems. Precision livestock farming technologies provide individualised monitoring of animals on commercial farms, optimising livestock production. Continuous acoustic monitoring is a widely accepted sensing technique used to estimate the daily rumination and grazing time budget of free-ranging cattle. However, typical environmental and natural noises on pasture noticeably affect the performance and generalisation of current acoustic methods. In this study, we present an acoustic method called Noise-Robust Foraging Activity Recognizer (NRFAR). The proposed method determines foraging activity bouts by analysing fixed-length segments of identified jaw movement events associated with grazing and rumination. The additive noise robustness of NRFAR was evaluated for several signal-to-noise ratios, using stationary Gaussian white noise and four different non-stationary natural noise sources. In noiseless conditions, NRFAR reaches an average balanced accuracy of 89%, outperforming two previous acoustic methods by more than 7%. Additionally, NRFAR presents better performance than previous acoustic methods in 66 out of 80 evaluated noisy scenarios (p<0.01). NRFAR operates online with a similar computational cost to previous acoustic methods. The combination of these properties and the high performance in harsh free-ranging environments render NRFAR an excellent choice for real-time implementation in a low-power embedded device. The instrumentation and computational algorithms presented within this publication are protected by a pending patent application: AR P20220100910. Web demo available at: https: sinc.unl.edu.ar web-demo nrfar
【4】 Deep Transfer Learning for Automatic Speech Recognition: Towards Better  Generalization标题：用于自动语音识别的深度迁移学习：更好的泛化链接：https://arxiv.org/abs/2304.14535
作者：Hamza Kheddar,Yassine Himeur,Somaya Al-Maadeed,Abbes Amira,Faycal Bensaali机构：LSEA Laboratory, Electrical Engineering Department, University of Medea, Algeria, College of Engineering and Information Technology, University of Dubai, Dubai, UAE, Department of Computer Science, University of Sharjah, UAE摘要：自动语音识别（ASR）最近已经成为使用深度学习（DL）时的一个重要挑战。它需要大规模的训练数据集和高计算和存储资源。此外，DL技术和机器学习（ML）方法通常假设训练和测试数据来自相同的域，具有相同的输入特征空间和数据分布特性。然而，这种假设在一些现实世界的人工智能（AI）应用中是不适用的。此外，存在收集真实数据具有挑战性、昂贵或很少发生的情况，这不能满足DL模型的数据要求。深度迁移学习（DTL）已被引入以克服这些问题，这有助于使用较小或略有不同但与训练数据相关的真实数据集开发高性能模型。本文介绍了一个全面的调查基于DTL的ASR框架，以阐明最新的发展，并帮助学者和专业人士了解当前的挑战。具体而言，介绍DTL背景后，采用了精心设计的分类法，以告知国家的最先进的。然后进行批判性分析，以确定每个框架的局限性和优势。接下来，引入了一项比较研究，以突出当前的挑战，然后为未来的研究提供机会。摘要：Automatic speech recognition (ASR) has recently become an important challenge when using deep learning (DL). It requires large-scale training datasets and high computational and storage resources. Moreover, DL techniques and machine learning (ML) approaches in general, hypothesize that training and testing data come from the same domain, with the same input feature space and data distribution characteristics. This assumption, however, is not applicable in some real-world artificial intelligence (AI) applications. Moreover, there are situations where gathering real data is challenging, expensive, or rarely occurring, which can not meet the data requirements of DL models. deep transfer learning (DTL) has been introduced to overcome these issues, which helps develop high-performing models using real datasets that are small or slightly different but related to the training data. This paper presents a comprehensive survey of DTL-based ASR frameworks to shed light on the latest developments and helps academics and professionals understand current challenges. Specifically, after presenting the DTL background, a well-designed taxonomy is adopted to inform the state-of-the-art. A critical analysis is then conducted to identify the limitations and advantages of each framework. Moving on, a comparative study is introduced to highlight the current challenges before deriving opportunities for future research.
【5】 Understanding Shared Speech-Text Representations标题：理解共享语音-文本表示法链接：https://arxiv.org/abs/2304.14514
作者：Gary Wang,Kyle Kastner,Ankur Bapna,Zhehuai Chen,Andrew Rosenberg,Bhuvana Ramabhadran,Yu Zhang机构：Google LLC, USA备注：Accepted at ICASSP 2023, camera ready摘要：最近，已经开发了许多通过将文本合并到端到端模型中来训练语音模型的方法，Mae-stro推进了最先进的自动语音识别（ASR）和语音翻译（ST）性能。在本文中，我们扩展了我们的理解所产生的共享语音文本representationwith两种类型的分析。首先，我们研究了语音自由域适应的限制，发现语料库特定的持续时间模型的语音文本对齐是学习共享的语音文本表示的最重要的组成部分。其次，我们检查的similarities激活的单峰（语音或文本）encodersas相比，一个共享的编码器的激活。我们发现，theshared编码器学习一个更紧凑和重叠的语音文本表示比单峰编码器。我们假设这部分解释了Maestro共享语音文本演示的有效性。摘要：Recently, a number of approaches to train speech models by incorpo-rating text into end-to-end models have been developed, with Mae-stro advancing state-of-the-art automatic speech recognition (ASR)and Speech Translation (ST) performance. In this paper, we expandour understanding of the resulting shared speech-text representationswith two types of analyses. First we examine the limits of speech-free domain adaptation, finding that a corpus-specific duration modelfor speech-text alignment is the most important component for learn-ing a shared speech-text representation. Second, we inspect the sim-ilarities between activations of unimodal (speech or text) encodersas compared to the activations of a shared encoder. We find that theshared encoder learns a more compact and overlapping speech-textrepresentation than the uni-modal encoders. We hypothesize that thispartially explains the effectiveness of the Maestro shared speech-textrepresentations.
【6】 Deep sound-field denoiser: optically-measured sound-field denoising  using deep neural network标题：深度声场去噪器：基于深度神经网络的光学测量声场去噪链接：https://arxiv.org/abs/2304.14923
作者：Kenji Ishikawa,Daiki Takeuchi,Noboru Harada,Takehiro Moriya机构：NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Atsugi, Kanagawa ,-, Japan备注：13 pages, 8 figures, 2 tables摘要：本文提出了一种深度声场去噪器，一种基于深度神经网络（DNN）的光学测量声场图像去噪。使用光学方法的声场成像由于其能够实现传统声学传感器无法实现的声学现象的高空间分辨率成像而获得了相当大的关注。然而，光学测量的声场图像往往严重污染的噪声，因为光学干涉测量的低灵敏度空气传播声。在这里，我们提出了一种基于DNN的声场去噪方法。时变声场图像序列通过使用时间方向傅里叶变换分解成谐波复振幅图像。复图像被转换成由实部和虚部组成的双通道图像，并通过非线性无激活网络去噪。该网络是训练的声场数据集从数值声学模拟随机参数。我们比较了传统的方法，如图像滤波器和时空滤波器，数值和实验数据。用平行相移干涉法和全息散斑干涉法测量了实验数据。所提出的深声场降噪显着优于传统的方法的数值和实验数据。摘要：This paper proposes a deep sound-field denoiser, a deep neural network (DNN) based denoising of optically measured sound-field images. Sound-field imaging using optical methods has gained considerable attention due to its ability to achieve high-spatial-resolution imaging of acoustic phenomena that conventional acoustic sensors cannot accomplish. However, the optically measured sound-field images are often heavily contaminated by noise because of the low sensitivity of optical interferometric measurements to airborne sound. Here, we propose a DNN-based sound-field denoising method. Time-varying sound-field image sequences are decomposed into harmonic complex-amplitude images by using a time-directional Fourier transform. The complex images are converted into two-channel images consisting of real and imaginary parts and denoised by a nonlinear-activation-free network. The network is trained on a sound-field dataset obtained from numerical acoustic simulations with randomized parameters. We compared the method with conventional ones, such as image filters and a spatiotemporal filter, on numerical and experimental data. The experimental data were measured by parallel phase-shifting interferometry and holographic speckle interferometry. The proposed deep sound-field denoiser significantly outperformed the conventional methods on both the numerical and experimental data.
eess.AS音频处理
【1】 Deep sound-field denoiser: optically-measured sound-field denoising  using deep neural network标题：深度声场去噪器：基于深度神经网络的光学测量声场去噪链接：https://arxiv.org/abs/2304.14923*与cs.SD语音【6】为同一篇
作者：Kenji Ishikawa,Daiki Takeuchi,Noboru Harada,Takehiro Moriya机构：NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Atsugi, Kanagawa ,-, Japan备注：13 pages, 8 figures, 2 tables摘要：本文提出了一种深度声场去噪器，一种基于深度神经网络（DNN）的光学测量声场图像去噪。使用光学方法的声场成像由于其能够实现传统声学传感器无法实现的声学现象的高空间分辨率成像而获得了相当大的关注。然而，光学测量的声场图像往往严重污染的噪声，因为光学干涉测量的低灵敏度空气传播声。在这里，我们提出了一种基于DNN的声场去噪方法。时变声场图像序列通过使用时间方向傅里叶变换分解成谐波复振幅图像。复图像被转换成由实部和虚部组成的双通道图像，并通过非线性无激活网络去噪。该网络是训练的声场数据集从数值声学模拟随机参数。我们比较了传统的方法，如图像滤波器和时空滤波器，数值和实验数据。用平行相移干涉法和全息散斑干涉法测量了实验数据。所提出的深声场降噪显着优于传统的方法的数值和实验数据。摘要：This paper proposes a deep sound-field denoiser, a deep neural network (DNN) based denoising of optically measured sound-field images. Sound-field imaging using optical methods has gained considerable attention due to its ability to achieve high-spatial-resolution imaging of acoustic phenomena that conventional acoustic sensors cannot accomplish. However, the optically measured sound-field images are often heavily contaminated by noise because of the low sensitivity of optical interferometric measurements to airborne sound. Here, we propose a DNN-based sound-field denoising method. Time-varying sound-field image sequences are decomposed into harmonic complex-amplitude images by using a time-directional Fourier transform. The complex images are converted into two-channel images consisting of real and imaginary parts and denoised by a nonlinear-activation-free network. The network is trained on a sound-field dataset obtained from numerical acoustic simulations with randomized parameters. We compared the method with conventional ones, such as image filters and a spatiotemporal filter, on numerical and experimental data. The experimental data were measured by parallel phase-shifting interferometry and holographic speckle interferometry. The proposed deep sound-field denoiser significantly outperformed the conventional methods on both the numerical and experimental data.
【2】 The ACM Multimedia 2023 Computational Paralinguistics Challenge: Emotion  Share & Requests标题：ACM多媒体2023计算副语言学挑战赛：情感分享与请求链接：https://arxiv.org/abs/2304.14882
*与cs.SD语音【1】为同一篇
作者：Björn W. Schuller,Anton Batliner,Shahin Amiriparian,Alexander Barnhill,Maurice Gerczuk,Andreas Triantafyllopoulos,Alice Baird,Panagiotis Tzirakis,Chris Gagne,Alan S. Cowen,Nikola Lackovic,Marie-José Caraty机构：GLAM, Imperial College London, London, United Kingdom, EIHW, University of Augsburg, Augsburg, Germany, FAU, Erlangen-Nuremberg, Germany, Hume AI, New York, USA, Malakoff Humanis, Paris, France, Sorbonne University, Claude Montacié备注：5 pages, part of the ACM Multimedia 2023 Grand Challenge "The ACM Multimedia 2023 Computational Paralinguistics Challenge (ComParE 2023). arXiv admin note: text overlap with arXiv:2205.06799摘要：ACM Multimedia 2023 Computational Paralinguistics Challenge首次在明确定义的条件下在研究竞赛中解决了两个不同的问题：在情感分享子挑战中，必须对言语进行回归;在请求子质询中，需要检测请求和投诉。我们描述了子挑战、基线特征提取和基于通常的CompPaRE特征的分类器、auDeep工具包以及使用DeepSpectRum工具包从预训练的CNN中提取深度特征;此外，使用wav 2 vec 2模型。摘要：The ACM Multimedia 2023 Computational Paralinguistics Challenge addresses two different problems for the first time in a research competition under well-defined conditions: In the Emotion Share Sub-Challenge, a regression on speech has to be made; and in the Requests Sub-Challenges, requests and complaints need to be detected. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the usual ComPaRE features, the auDeep toolkit, and deep feature extraction from pre-trained CNNs using the DeepSpectRum toolkit; in addition, wav2vec2 models are used.
【3】 Musical Voice Separation as Link Prediction: Modeling a Musical  Perception Task as a Multi-Trajectory Tracking Problem标题：作为链接预测的音乐声音分离：将音乐感知任务建模为多轨迹跟踪问题链接：https://arxiv.org/abs/2304.14848
*与cs.SD语音【2】为同一篇
作者：Emmanouil Karystinaios,Francesco Foscarin,Gerhard Widmer机构：Johannes Kepler University Linz, Austria, LIT AI Lab, Linz Institute of Technology, Austria, † Equal contribution among authors.备注：Accepted at the 32nd International Joint Conference on Artificial Intelligence (IJCAI-23)摘要：本文的目标是分离不同的交互声音的知觉任务，即。例如，在一个复调音乐作品中的单声道旋律流。我们的目标是符号音乐，音符被明确编码，并将此任务建模为离散观测的多轨迹跟踪（MTT）问题。例如，在一个音高时间空间中的音符。我们的方法从一个音乐作品，通过创建一个节点为每个音符，并通过预测两个音符之间的链接，如果他们是连续的在同一个声音 流分离的旋律轨迹。这种局部的贪婪预测是可能的节点嵌入创建的异构图神经网络，可以捕获内部和内部轨迹信息。此外，我们提出了一个新的正则化损失，鼓励输出尊重MTT的前提下，最多一个传入和一个传出链路的每个节点，有利于单声道（语音）轨迹;该损失函数在其它一般MTT情形中也可能是有用的。我们的方法不使用特定领域的启发式，可扩展到更长的序列和更高数量的声音，并可以处理复杂的情况下，如语音反转和重叠。我们达到新的国家的最先进的结果，在不同风格的古典音乐的声音分离任务。摘要：This paper targets the perceptual task of separating the different interacting voices, i.e., monophonic melodic streams, in a polyphonic musical piece. We target symbolic music, where notes are explicitly encoded, and model this task as a Multi-Trajectory Tracking (MTT) problem from discrete observations, i.e., notes in a pitch-time space. Our approach builds a graph from a musical piece, by creating one node for every note, and separates the melodic trajectories by predicting a link between two notes if they are consecutive in the same voice stream. This kind of local, greedy prediction is made possible by node embeddings created by a heterogeneous graph neural network that can capture inter- and intra-trajectory information. Furthermore, we propose a new regularization loss that encourages the output to respect the MTT premise of at most one incoming and one outgoing link for every node, favouring monophonic (voice) trajectories; this loss function might also be useful in other general MTT scenarios. Our approach does not use domain-specific heuristics, is scalable to longer sequences and a higher number of voices, and can handle complex cases such as voice inversions and overlaps. We reach new state-of-the-art results for the voice separation task in classical music of different styles.
【4】 A noise-robust acoustic method for recognition of foraging activities of  grazing cattle标题：一种识别放牧牛觅食行为的抗噪声学方法链接：https://arxiv.org/abs/2304.14824
*与cs.SD语音【3】为同一篇
作者：Luciano S. Martinez-Rau,José O. Chelotti,Mariano Ferrero,Julio R. Galli,Santiago A. Utsumi,Alejandra M. Planisich,H. Leonardo Rufiner,Leonardo L. Giovanini机构：Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i), FICH-UNLCONICET, Argentina, Department of Computer and Electrical Engineering, Mid Sweden University Sundsvall备注：list of used audio-clips is available in the list_audio_clips.xlsx摘要：为了在不断增长的乳制品市场中保持竞争力，农民必须不断改进他们的畜牧生产系统。精准畜牧技术为商业农场提供个性化的动物监测，优化畜牧生产。连续声学监测是一种被广泛接受的传感技术，用于估计自由放养牛的每日反刍和放牧时间预算。然而，牧场上典型的环境和自然噪声显著影响当前声学方法的性能和泛化。在这项研究中，我们提出了一种声学方法称为噪声鲁棒觅食活动识别器（NRFAR）。所提出的方法确定觅食活动回合通过分析固定长度段确定的下颌运动事件与放牧和反刍。使用平稳高斯白噪声和四种不同的非平稳自然噪声源，NRFAR的加性噪声鲁棒性进行了评估的几个信号-噪声比。在无噪声条件下，NRFAR达到了89%的平均平衡精度，比以前的两种声学方法高出7%以上。此外，NRFAR在80个评估的噪声场景中的66个中呈现出比先前声学方法更好的性能（p〈0.01）。NRFAR在线操作，具有与先前声学方法类似的计算成本。这些特性的组合和在恶劣的自由测距环境中的高性能使NRFAR成为在低功耗嵌入式设备中实时实现的绝佳选择。本出版物中呈现的仪器和计算算法受未决专利申请的保护：AR P20220100910。Web演示网址：www.example.com摘要：To stay competitive in the growing dairy market, farmers must continuously improve their livestock production systems. Precision livestock farming technologies provide individualised monitoring of animals on commercial farms, optimising livestock production. Continuous acoustic monitoring is a widely accepted sensing technique used to estimate the daily rumination and grazing time budget of free-ranging cattle. However, typical environmental and natural noises on pasture noticeably affect the performance and generalisation of current acoustic methods. In this study, we present an acoustic method called Noise-Robust Foraging Activity Recognizer (NRFAR). The proposed method determines foraging activity bouts by analysing fixed-length segments of identified jaw movement events associated with grazing and rumination. The additive noise robustness of NRFAR was evaluated for several signal-to-noise ratios, using stationary Gaussian white noise and four different non-stationary natural noise sources. In noiseless conditions, NRFAR reaches an average balanced accuracy of 89%, outperforming two previous acoustic methods by more than 7%. Additionally, NRFAR presents better performance than previous acoustic methods in 66 out of 80 evaluated noisy scenarios (p<0.01). NRFAR operates online with a similar computational cost to previous acoustic methods. The combination of these properties and the high performance in harsh free-ranging environments render NRFAR an excellent choice for real-time implementation in a low-power embedded device. The instrumentation and computational algorithms presented within this publication are protected by a pending patent application: AR P20220100910. Web demo available at: https: sinc.unl.edu.ar web-demo nrfar
【5】 Deep Transfer Learning for Automatic Speech Recognition: Towards Better  Generalization标题：用于自动语音识别的深度迁移学习：更好的泛化链接：https://arxiv.org/abs/2304.14535*与cs.SD语音【4】为同一篇
作者：Hamza Kheddar,Yassine Himeur,Somaya Al-Maadeed,Abbes Amira,Faycal Bensaali机构：LSEA Laboratory, Electrical Engineering Department, University of Medea, Algeria, College of Engineering and Information Technology, University of Dubai, Dubai, UAE, Department of Computer Science, University of Sharjah, UAE摘要：自动语音识别（ASR）最近已经成为使用深度学习（DL）时的一个重要挑战。它需要大规模的训练数据集和高计算和存储资源。此外，DL技术和机器学习（ML）方法通常假设训练和测试数据来自相同的域，具有相同的输入特征空间和数据分布特性。然而，这种假设在一些现实世界的人工智能（AI）应用中是不适用的。此外，存在收集真实数据具有挑战性、昂贵或很少发生的情况，这不能满足DL模型的数据要求。深度迁移学习（DTL）已被引入以克服这些问题，这有助于使用较小或略有不同但与训练数据相关的真实数据集开发高性能模型。本文介绍了一个全面的调查基于DTL的ASR框架，以阐明最新的发展，并帮助学者和专业人士了解当前的挑战。具体而言，介绍DTL背景后，采用了精心设计的分类法，以告知国家的最先进的。然后进行批判性分析，以确定每个框架的局限性和优势。接下来，引入了一项比较研究，以突出当前的挑战，然后为未来的研究提供机会。摘要：Automatic speech recognition (ASR) has recently become an important challenge when using deep learning (DL). It requires large-scale training datasets and high computational and storage resources. Moreover, DL techniques and machine learning (ML) approaches in general, hypothesize that training and testing data come from the same domain, with the same input feature space and data distribution characteristics. This assumption, however, is not applicable in some real-world artificial intelligence (AI) applications. Moreover, there are situations where gathering real data is challenging, expensive, or rarely occurring, which can not meet the data requirements of DL models. deep transfer learning (DTL) has been introduced to overcome these issues, which helps develop high-performing models using real datasets that are small or slightly different but related to the training data. This paper presents a comprehensive survey of DTL-based ASR frameworks to shed light on the latest developments and helps academics and professionals understand current challenges. Specifically, after presenting the DTL background, a well-designed taxonomy is adopted to inform the state-of-the-art. A critical analysis is then conducted to identify the limitations and advantages of each framework. Moving on, a comparative study is introduced to highlight the current challenges before deriving opportunities for future research.
【6】 Understanding Shared Speech-Text Representations标题：理解共享语音-文本表示法链接：https://arxiv.org/abs/2304.14514*与cs.SD语音【5】为同一篇
作者：Gary Wang,Kyle Kastner,Ankur Bapna,Zhehuai Chen,Andrew Rosenberg,Bhuvana Ramabhadran,Yu Zhang机构：Google LLC, USA备注：Accepted at ICASSP 2023, camera ready摘要：最近，已经开发了许多通过将文本合并到端到端模型中来训练语音模型的方法，Mae-stro推进了最先进的自动语音识别（ASR）和语音翻译（ST）性能。在本文中，我们扩展了我们的理解所产生的共享语音文本representationwith两种类型的分析。首先，我们研究了语音自由域适应的限制，发现语料库特定的持续时间模型的语音文本对齐是学习共享的语音文本表示的最重要的组成部分。其次，我们检查的similarities激活的单峰（语音或文本）encodersas相比，一个共享的编码器的激活。我们发现，theshared编码器学习一个更紧凑和重叠的语音文本表示比单峰编码器。我们假设这部分解释了Maestro共享语音文本演示的有效性。摘要：Recently, a number of approaches to train speech models by incorpo-rating text into end-to-end models have been developed, with Mae-stro advancing state-of-the-art automatic speech recognition (ASR)and Speech Translation (ST) performance. In this paper, we expandour understanding of the resulting shared speech-text representationswith two types of analyses. First we examine the limits of speech-free domain adaptation, finding that a corpus-specific duration modelfor speech-text alignment is the most important component for learn-ing a shared speech-text representation. Second, we inspect the sim-ilarities between activations of unimodal (speech or text) encodersas compared to the activations of a shared encoder. We find that theshared encoder learns a more compact and overlapping speech-textrepresentation than the uni-modal encoders. We hypothesize that thispartially explains the effectiveness of the Maestro shared speech-textrepresentations.
机器翻译，仅供参考