今日论文合集：cs.SD语音9篇，eess.AS音频处理10篇。
cs.SD语音
【1】 Musical Excellence of Mridangam: an introductory review标题：《Mriangam》的音乐卓越：介绍性评论链接：https://arxiv.org/abs/2307.09425
作者：Arvind Shankar Kumar机构：arXiv:,.,v,  [cs.SD]  , Jul摘要：这是一个介绍性的审查音乐卓越的Mridangam博士。Umayalpuram K Sivaraman博士T Ramasami和Dr. Naresh，这是一本科学论文，探索了古印度古典打击乐器Mridangam的独特音调特性。这篇评论旨在弥合Mridangam的音乐卓越的主要目标受众之间的差距-听众，艺术家和制造商-和原始论文的科学严谨性，首先介绍音乐分析的概念，然后介绍和解释在此背景下所做的发现。本综述的前三章介绍了Mridangam的音乐卓越中使用的基本科学概念，并提供了以前对这种乐器的科学研究背景，从博士的开创性工作开始。CV拉曼。这也包括对《Mridangam的音乐卓越》中相应章节的简要讨论。接下来的章节都服务于解释主要的科学成果提出的音乐卓越的Mridangam在每一个相应的章节在论文中，并最终总结了相关的工作。摘要：This is an introductory review of Musical Excellence of Mridangam by Dr. Umayalpuram K Sivaraman, Dr. T Ramasami and Dr. Naresh, which is a scientific treatise exploring the unique tonal properties of the ancient Indian classical percussive instrument -- the Mridangam. This review aims to bridge the gap between the primary intended audience of Musical Excellence of Mridangam - listeners, artistes and makers -- and the scientific rigour with which the original treatise is written, by first introducing the concepts of musical analysis and then presenting and explaining the discoveries made within this context. The first three chapters of this review introduce the basic scientific concepts used in Musical Excellence of Mridangam and provides background to previous scientific research into this instrument, starting from the seminal work of Dr. CV Raman. This also includes brief discussions of the corresponding chapters in Musical Excellence of Mridangam. The next chapters all serve the purpose of explaining the main scientific results presented in Musical Excellence of Mridangam in each of the corresponding chapters in the treatise, and finally summarizing the relevance of the work.
【2】 Adapting an ASR Foundation Model for Spoken Language Assessment标题：采用ASR基础模式进行口语评估链接：https://arxiv.org/abs/2307.09378
作者：Rao Ma,Mengjie Qian,Mark J. F. Gales,Kate M. Knill机构：ALTA Institute, Machine Intelligence Lab, Department of Engineering, Cambridge University, UK备注：Submitted to SLaTE 2023摘要：准确可靠的口语评估系统的关键部分是底层的ASR模型。最近，大规模的预训练ASR基础模型，如Whisper已经可用。由于这些模型的输出被设计为人类可读，因此添加了标点符号，数字以阿拉伯数字形式呈现，并包括缩写。此外，这些模型有一种倾向，跳过输出中的不流利和犹豫。虽然这些属性有助于可读性，但对于评估候选人的能力和提供反馈没有帮助。这里需要准确地记录候选人所说的内容。在本文中，我们对Whisper输出进行了详细分析，并提出了两种解决方案：微调和软提示调谐。实验进行了公共演讲语料库和英语学习者数据集。结果表明，我们可以有效地改变Whisper的解码行为，以生成确切的话说的响应。摘要：A crucial part of an accurate and reliable spoken language assessment system is the underlying ASR model. Recently, large-scale pre-trained ASR foundation models such as Whisper have been made available. As the output of these models is designed to be human readable, punctuation is added, numbers are presented in Arabic numeric form and abbreviations are included. Additionally, these models have a tendency to skip disfluencies and hesitations in the output. Though useful for readability, these attributes are not helpful for assessing the ability of a candidate and providing feedback. Here a precise transcription of what a candidate said is needed. In this paper, we give a detailed analysis of Whisper outputs and propose two solutions: fine-tuning and soft prompt tuning. Experiments are conducted on both public speech corpora and an English learner dataset. Results show that we can effectively alter the decoding behaviour of Whisper to generate the exact words spoken in the response.
【3】 FlexiAST: Flexibility is What AST Needs标题：FlexiAST：灵活性是AST所需要的链接：https://arxiv.org/abs/2307.09286
作者：Jiu Feng,Mehmet Hamza Erol,Joon Son Chung,Arda Senocak机构：Korea Advanced Institute of Science and Technology, South Korea备注：Interspeech 2023摘要：这项工作的目的是给补丁大小的灵活性，音频频谱图Transformers（AST）。AST的最新进展已经在各种基于音频的任务中显示出优异的性能。然而，标准的AST的性能急剧下降时，使用不同的补丁大小从训练过程中使用的评估。因此，AST模型通常被重新训练以适应补丁大小的变化。为了克服这一限制，本文提出了一个训练过程，以提供灵活性，标准AST模型，而无需架构的变化，使他们能够在推理阶段的各种补丁大小- FlexiAST。这种训练方法简单地利用随机补丁大小选择和补丁和位置嵌入权重的大小调整。我们的实验表明，FlexiAST提供了类似的性能标准AST模型，同时保持其评估能力，在不同的补丁大小在不同的数据集上的音频分类任务。摘要：The objective of this work is to give patch-size flexibility to Audio Spectrogram Transformers (AST). Recent advancements in ASTs have shown superior performance in various audio-based tasks. However, the performance of standard ASTs degrades drastically when evaluated using different patch sizes from that used during training. As a result, AST models are typically re-trained to accommodate changes in patch sizes. To overcome this limitation, this paper proposes a training procedure to provide flexibility to standard AST models without architectural changes, allowing them to work with various patch sizes at the inference stage - FlexiAST. This proposed training approach simply utilizes random patch size selection and resizing of patch and positional embedding weights. Our experiments show that FlexiAST gives similar performance to standard AST models while maintaining its evaluation ability at various patch sizes on different datasets for audio classification tasks.
【4】 Detecting Throat Cancer from Speech Signals Using Machine Learning: A  Reproducible Literature Review标题：利用机器学习从语音信号中检测喉癌：一篇可重复的文献综述链接：https://arxiv.org/abs/2307.09230
作者：Mary Paterson,James Moor,Luisa Cutillo机构：University of Leeds, Leeds Teaching Hospitals NHS Trust备注：19 pages, 10 figures摘要：在这项工作中，我们对使用机器学习和人工智能从语音记录中检测喉癌的现有文献进行了范围审查。我们发现在这一领域的22篇论文，并讨论了他们的方法和结果。我们将这些文件分成两组-九个执行二进制分类，和13个执行多类分类。这些论文提出了一系列最常用的神经网络方法。在分类之前，还从音频中提取了许多特征，其中最常见的是梅尔频率倒谱系数。本次检索中发现的论文均没有相关的代码存储库，因此不可复制。因此，我们创建了一个公开可用的代码库我们自己的分类器。我们使用迁移学习的多类问题，分类三个病理和健康对照。使用这种技术，我们实现了53.54%的未加权平均召回率，83.14%的敏感性，64.00%的特异性。我们比较我们的分类器与相同的数据集上获得的结果，并发现类似的结果。摘要：In this work we perform a scoping review of the current literature on the detection of throat cancer from speech recordings using machine learning and artificial intelligence. We find 22 papers within this area and discuss their methods and results. We split these papers into two groups - nine performing binary classification, and 13 performing multi-class classification. The papers present a range of methods with neural networks being most commonly implemented. Many features are also extracted from the audio before classification, with the most common bring mel-frequency cepstral coefficients. None of the papers found in this search have associated code repositories and as such are not reproducible. Therefore, we create a publicly available code repository of our own classifiers. We use transfer learning on a multi-class problem, classifying three pathologies and healthy controls. Using this technique we achieve an unweighted average recall of 53.54%, sensitivity of 83.14%, and specificity of 64.00%. We compare our classifiers with the results obtained on the same dataset and find similar results.
【5】 OxfordVGG Submission to the EGO4D AV Transcription Challenge标题：OxfordVGG提交给EGO4D AV转录挑战赛链接：https://arxiv.org/abs/2307.09006
作者：Jaesung Huh,Max Bain,Andrew Zisserman机构：Visual Geometry Group, University of Oxford, UK备注：Technical Report摘要：本报告介绍了我们提交的牛津VGG团队EGO4D视听（AV）自动语音识别挑战赛2023的技术细节。我们提出了WhisperX，一个系统，高效的语音转录的长格式音频与字级时间对齐，以及两个文本规范这是公开可用的。我们最终提交的作品在挑战测试集上获得了56.0%的单词错误率（WER），在排行榜上排名第一。所有基准代码和模型均可在https://github.com/m-bain/whisperX上获得。摘要：This report presents the technical details of our submission on the EGO4D Audio-Visual (AV) Automatic Speech Recognition Challenge 2023 from the OxfordVGG team. We present WhisperX, a system for efficient speech transcription of long-form audio with word-level time alignment, along with two text normalisers which are publicly available. Our final submission obtained 56.0% of the Word Error Rate (WER) on the challenge test set, ranked 1st on the leaderboard. All baseline codes and models are available on https://github.com/m-bain/whisperX.
【6】 On Computing In the Network: Covid-19 Coughs Detection Case Study标题：论网络计算：新冠肺炎咳嗽检测案例研究链接：https://arxiv.org/abs/2307.08902
作者：Soukaina Ouledsidi Ali,Zakaria Ait Hmitti,Halima Elbiaze,Roch Glitho机构：Glitho,  Universit´e du Qu´ebec A Montr´eal, Montreal, Canada,  Concordia University, Montreal, Canada备注：None摘要：网络中的计算（COIN）是一种有前途的技术，其允许在诸如交换机和网络接口卡之类的网络设备内执行处理。时间敏感型应用可以通过在云-边缘-雾连续体中灵活地分配缓存和计算任务来实现其服务质量（QoS）目标。本文强调了网络计算与边缘计算相比在延迟和流量过滤方面的优势。我们考虑了一个与机场环境中的Covid-19警报应用相关的关键用例。通过咳嗽分析监测到达的旅客，以便发现潜在的感染病例并进行隔离进行医学检查。已经在使用网络内计算的架构和使用边缘计算的架构之间进行了性能比较。我们使用模拟表明，在往返时间（RTT）和流量过滤方面，网络内计算优于边缘计算。摘要：Computing in the network (COIN) is a promising technology that allows processing to be carried out within network devices such as switches and network interface cards. Time sensitive application can achieve their quality of service (QoS) target by flexibly distributing the caching and computing tasks in the cloud-edge-mist continuum. This paper highlights the advantages of in-network computing, comparing to edge computing, in terms of latency and traffic filtering. We consider a critical use case related to Covid-19 alert application in an airport setting. Arriving travelers are monitored through cough analysis so that potentially infected cases can be detected and isolated for medical tests. A performance comparison has been done between an architecture using in-network computing and another one using edge computing. We show using simulations that in-network computing outperforms edge computing in terms of Round Trip Time (RTT) and traffic filtering.
【7】 SLMGAN: Exploiting Speech Language Model Representations for  Unsupervised Zero-Shot Voice Conversion in GANs标题：SLMGAN：在GANS中利用语音语言模型表示进行无监督零发声转换链接：https://arxiv.org/abs/2307.09435
作者：Yinghao Aaron Li,Cong Han,Nima Mesgarani机构：Department of Electrical Engineering, Columbia University, USA备注：WASPAA 2023摘要：近年来，大规模预训练语音语言模型（SLM）在各种生成语音建模应用中取得了显着的进步，例如文本到语音合成，语音转换和语音增强。这些应用通常涉及将文本或语音输入映射到预训练的SLM表示，从中解码目标语音。本文介绍了一种新的方法，SLMGAN，利用SLM表示的生成对抗网络（GAN）框架内的判别任务，特别是语音转换。在StarGANv 2-VC的基础上，我们在基于mel的判别器之上添加了我们新的基于SLM的WavLM判别器，以及我们新设计的SLM特征匹配损失函数，从而产生了一个无监督zero-shot语音转换系统，在训练过程中不需要文本标签。主观评价结果表明，SLMGAN优于现有的国家的最先进的zero-shot语音转换模型的自然度，并实现了可比的相似性，突出了潜在的SLM为基础的歧视，为相关的应用。摘要：In recent years, large-scale pre-trained speech language models (SLMs) have demonstrated remarkable advancements in various generative speech modeling applications, such as text-to-speech synthesis, voice conversion, and speech enhancement. These applications typically involve mapping text or speech inputs to pre-trained SLM representations, from which target speech is decoded. This paper introduces a new approach, SLMGAN, to leverage SLM representations for discriminative tasks within the generative adversarial network (GAN) framework, specifically for voice conversion. Building upon StarGANv2-VC, we add our novel SLM-based WavLM discriminators on top of the mel-based discriminators along with our newly designed SLM feature matching loss function, resulting in an unsupervised zero-shot voice conversion system that does not require text labels during training. Subjective evaluation results show that SLMGAN outperforms existing state-of-the-art zero-shot voice conversion models in terms of naturalness and achieves comparable similarity, highlighting the potential of SLM-based discriminators for related applications.
【8】 Efficient representation of head-related transfer functions in  continuous space-frequency domains标题：头部相关传递函数在连续空间-频域的有效表示链接：https://arxiv.org/abs/2307.09352
作者：Adam Szwajcowski机构：AGH University of Science and Technology, Department of Robotics and Mechatronics备注：None摘要：利用球谐（SH）域已被建立为默认的方法，获得连续的头部相关的传递函数（HRTF）在空间上。本文涉及不同的变种，扩展这个解决方案，取代SH与四维（4D）连续的功能模型，其中频率被想象为另一个物理维度。最近开发的超球谐（HSH）表示相比，在球柱坐标系中定义的模型合并SH与一维基函数。这两种方法的效率进行评估的基础上，从HUTUBS数据库，包括详细分析其依赖于所选择的频率和空间的近似阶数的个别HRTF的再现误差。采用在4D坐标系中定义的连续函数模型允许HRTF幅度谱被表达为一小组系数，其可以被解码回任何方向和频率的值。HSH和SH与反向傅里叶-贝塞尔级数合并的性能最好，前者具有更好的压缩能力，对于低数量的系数实现略高的精度。所提出的模型可以用于多种目的，诸如用于机器学习应用的内插、压缩或参数化，并且不仅可以应用于HRTF，而且还可以应用于其他类型的方向性函数，例如，HRTF。声源指向性摘要：Utilizing spherical harmonic (SH) domain has been established as the default method of obtaining continuity over space in head-related transfer functions (HRTFs). This paper concerns different variants of extending this solution by replacing SHs with four-dimensional (4D) continuous functional models in which frequency is imagined as another physical dimension. Recently developed hyperspherical harmonic (HSH) representation is compared with models defined in spherindrical coordinate system by merging SHs with one-dimensional basis functions. The efficiency of both approaches is evaluated based on the reproduction errors for individual HRTFs from HUTUBS database, including detailed analysis of its dependency on chosen orders of approximation in frequency and space. Employing continuous functional models defined in 4D coordinate systems allows HRTF magnitude spectra to be expressed as a small set of coefficients which can be decoded back into values at any direction and frequency. The best performance was noted for HSHs and SHs merged with reverse Fourier-Bessel series, with the former featuring better compression abilities, achieving slightly higher accuracy for low number of coefficients. The presented models can serve multiple purposes, such as interpolation, compression or parametrization for machine learning applications, and can be applied not only to HRTFs but also to other types of directivity functions, e.g. sound source directivity.
【9】 ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and  Development标题：Ivrit.ai：面向人工智能研究和开发的希伯来语语音综合数据集链接：https://arxiv.org/abs/2307.08720
作者：Yanir Marmor,Kinneret Misgav,Yair Lifshitz机构：Weizmann Institute of Science, Hadassah Research Fund备注：9 pages, 1 table and 3 figures摘要：我们介绍“ivrit.ai”，一个全面的希伯来语语音数据集，解决了明显缺乏广泛的，高质量的资源，以推进希伯来语的自动语音识别（ASR）技术。拥有超过3，300个演讲小时和超过一千个不同的演讲者，ivrit.ai提供了各种背景下的希伯来语演讲的大量汇编。它以三种形式提供，以满足不同的研究需求：未处理的原始音频;语音活动检测后的数据和部分转录的数据。该数据集以其合法可访问性而脱颖而出，允许免费使用，从而成为研究人员，开发人员和商业实体的重要资源。ivrit.ai开放了许多应用程序，为增强希伯来语的AI功能提供了巨大的潜力。未来的努力旨在进一步扩展ivrit.ai，从而提升希伯来语在人工智能研究和技术方面的地位。摘要：We introduce "ivrit.ai", a comprehensive Hebrew speech dataset, addressing the distinct lack of extensive, high-quality resources for advancing Automated Speech Recognition (ASR) technology in Hebrew. With over 3,300 speech hours and a over a thousand diverse speakers, ivrit.ai offers a substantial compilation of Hebrew speech across various contexts. It is delivered in three forms to cater to varying research needs: raw unprocessed audio; data post-Voice Activity Detection, and partially transcribed data. The dataset stands out for its legal accessibility, permitting use at no cost, thereby serving as a crucial resource for researchers, developers, and commercial entities. ivrit.ai opens up numerous applications, offering vast potential to enhance AI capabilities in Hebrew. Future efforts aim to expand ivrit.ai further, thereby advancing Hebrew's standing in AI research and technology.
eess.AS音频处理
【1】 SLMGAN: Exploiting Speech Language Model Representations for  Unsupervised Zero-Shot Voice Conversion in GANs标题：SLMGAN：在GANS中利用语音语言模型表示进行无监督零发声转换链接：https://arxiv.org/abs/2307.09435
作者：Yinghao Aaron Li,Cong Han,Nima Mesgarani机构：Department of Electrical Engineering, Columbia University, USA备注：WASPAA 2023摘要：近年来，大规模预训练语音语言模型（SLM）在各种生成语音建模应用中取得了显着的进步，例如文本到语音合成，语音转换和语音增强。这些应用通常涉及将文本或语音输入映射到预训练的SLM表示，从中解码目标语音。本文介绍了一种新的方法，SLMGAN，利用SLM表示的生成对抗网络（GAN）框架内的判别任务，特别是语音转换。在StarGANv 2-VC的基础上，我们在基于mel的判别器之上添加了我们新的基于SLM的WavLM判别器，以及我们新设计的SLM特征匹配损失函数，从而产生了一个无监督zero-shot语音转换系统，在训练过程中不需要文本标签。主观评价结果表明，SLMGAN优于现有的国家的最先进的zero-shot语音转换模型的自然度，并实现了可比的相似性，突出了潜在的SLM为基础的歧视，为相关的应用。摘要：In recent years, large-scale pre-trained speech language models (SLMs) have demonstrated remarkable advancements in various generative speech modeling applications, such as text-to-speech synthesis, voice conversion, and speech enhancement. These applications typically involve mapping text or speech inputs to pre-trained SLM representations, from which target speech is decoded. This paper introduces a new approach, SLMGAN, to leverage SLM representations for discriminative tasks within the generative adversarial network (GAN) framework, specifically for voice conversion. Building upon StarGANv2-VC, we add our novel SLM-based WavLM discriminators on top of the mel-based discriminators along with our newly designed SLM feature matching loss function, resulting in an unsupervised zero-shot voice conversion system that does not require text labels during training. Subjective evaluation results show that SLMGAN outperforms existing state-of-the-art zero-shot voice conversion models in terms of naturalness and achieves comparable similarity, highlighting the potential of SLM-based discriminators for related applications.
【2】 Efficient representation of head-related transfer functions in  continuous space-frequency domains标题：头部相关传递函数在连续空间-频域的有效表示链接：https://arxiv.org/abs/2307.09352
作者：Adam Szwajcowski机构：AGH University of Science and Technology, Department of Robotics and Mechatronics备注：None摘要：利用球谐（SH）域已被建立为默认的方法，获得连续的头部相关的传递函数（HRTF）在空间上。本文涉及不同的变种，扩展这个解决方案，取代SH与四维（4D）连续的功能模型，其中频率被想象为另一个物理维度。最近开发的超球谐（HSH）表示相比，在球柱坐标系中定义的模型合并SH与一维基函数。这两种方法的效率进行评估的基础上，从HUTUBS数据库，包括详细分析其依赖于所选择的频率和空间的近似阶数的个别HRTF的再现误差。采用在4D坐标系中定义的连续函数模型允许HRTF幅度谱被表达为一小组系数，其可以被解码回任何方向和频率的值。HSH和SH与反向傅里叶-贝塞尔级数合并的性能最好，前者具有更好的压缩能力，对于低数量的系数实现略高的精度。所提出的模型可以用于多种目的，诸如用于机器学习应用的内插、压缩或参数化，并且不仅可以应用于HRTF，而且还可以应用于其他类型的方向性函数，例如，HRTF。声源指向性摘要：Utilizing spherical harmonic (SH) domain has been established as the default method of obtaining continuity over space in head-related transfer functions (HRTFs). This paper concerns different variants of extending this solution by replacing SHs with four-dimensional (4D) continuous functional models in which frequency is imagined as another physical dimension. Recently developed hyperspherical harmonic (HSH) representation is compared with models defined in spherindrical coordinate system by merging SHs with one-dimensional basis functions. The efficiency of both approaches is evaluated based on the reproduction errors for individual HRTFs from HUTUBS database, including detailed analysis of its dependency on chosen orders of approximation in frequency and space. Employing continuous functional models defined in 4D coordinate systems allows HRTF magnitude spectra to be expressed as a small set of coefficients which can be decoded back into values at any direction and frequency. The best performance was noted for HSHs and SHs merged with reverse Fourier-Bessel series, with the former featuring better compression abilities, achieving slightly higher accuracy for low number of coefficients. The presented models can serve multiple purposes, such as interpolation, compression or parametrization for machine learning applications, and can be applied not only to HRTFs but also to other types of directivity functions, e.g. sound source directivity.
【3】 Low bit rate binaural link for improved ultra low-latency low-complexity  multichannel speech enhancement in Hearing Aids标题：用于改进助听器中超低延迟低复杂度多声道语音增强的低比特率双耳链路链接：https://arxiv.org/abs/2307.08858
作者：Nils L. Westhausen,Bernd T. Meyer机构：Communication Acoustics & Cluster of Excellence Hearing,all, Carl von Ossietzky University, Oldenburg, Germany备注：Accepted at WASPAA 2023摘要：助听器中的语音增强是一项具有挑战性的任务，因为硬件限制了可能操作的数量，并且延迟需要在仅几毫秒的范围内。我们提出了一个与这些限制兼容的深度学习模型，我们称之为组通信过滤求和网络（GCFSnet）。GCFSnet是一种因果多输入单输出增强模型，使用时频域中的滤波求和处理和多帧深度后置滤波器。所有过滤器都是复值的，并通过深度学习模型进行估计，通过组通信和量化感知训练使用权重共享，以减少模型大小和计算占用空间。为了进一步提高性能，提出了用于延迟双耳特征的低比特率双耳链路，以使用双耳信息，同时保留2ms的延迟。在非低延迟配置中的oracle双耳LCMV波束形成器的性能甚至可以在客观度量方面通过GCFSnet的单边配置来匹配。摘要：Speech enhancement in hearing aids is a challenging task since the hardware limits the number of possible operations and the latency needs to be in the range of only a few milliseconds. We propose a deep-learning model compatible with these limitations, which we refer to as Group-Communication Filter-and-Sum Network (GCFSnet). GCFSnet is a causal multiple-input single output enhancement model using filter-and-sum processing in the time-frequency domain and a multi-frame deep post filter. All filters are complex-valued and are estimated by a deep-learning model using weight-sharing through Group Communication and quantization-aware training for reducing model size and computational footprint. For a further increase in performance, a low bit rate binaural link for delayed binaural features is proposed to use binaural information while retaining a latency of 2ms. The performance of an oracle binaural LCMV beamformer in non-low-latency configuration can be matched even by a unilateral configuration of the GCFSnet in terms of objective metrics.
【4】 ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and  Development标题：Ivrit.ai：面向人工智能研究和开发的希伯来语语音综合数据集链接：https://arxiv.org/abs/2307.08720
作者：Yanir Marmor,Kinneret Misgav,Yair Lifshitz机构：Weizmann Institute of Science, Hadassah Research Fund备注：9 pages, 1 table and 3 figures摘要：我们介绍“ivrit.ai”，一个全面的希伯来语语音数据集，解决了明显缺乏广泛的，高质量的资源，以推进希伯来语的自动语音识别（ASR）技术。拥有超过3，300个演讲小时和超过一千个不同的演讲者，ivrit.ai提供了各种背景下的希伯来语演讲的大量汇编。它以三种形式提供，以满足不同的研究需求：未处理的原始音频;语音活动检测后的数据和部分转录的数据。该数据集以其合法可访问性而脱颖而出，允许免费使用，从而成为研究人员，开发人员和商业实体的重要资源。ivrit.ai开放了许多应用程序，为增强希伯来语的AI功能提供了巨大的潜力。未来的努力旨在进一步扩展ivrit.ai，从而提升希伯来语在人工智能研究和技术方面的地位。摘要：We introduce "ivrit.ai", a comprehensive Hebrew speech dataset, addressing the distinct lack of extensive, high-quality resources for advancing Automated Speech Recognition (ASR) technology in Hebrew. With over 3,300 speech hours and a over a thousand diverse speakers, ivrit.ai offers a substantial compilation of Hebrew speech across various contexts. It is delivered in three forms to cater to varying research needs: raw unprocessed audio; data post-Voice Activity Detection, and partially transcribed data. The dataset stands out for its legal accessibility, permitting use at no cost, thereby serving as a crucial resource for researchers, developers, and commercial entities. ivrit.ai opens up numerous applications, offering vast potential to enhance AI capabilities in Hebrew. Future efforts aim to expand ivrit.ai further, thereby advancing Hebrew's standing in AI research and technology.
【5】 Musical Excellence of Mridangam: an introductory review标题：《Mriangam》的音乐卓越：介绍性评论链接：https://arxiv.org/abs/2307.09425
作者：Arvind Shankar Kumar机构：arXiv:,.,v,  [cs.SD]  , Jul摘要：这是一个介绍性的审查音乐卓越的Mridangam博士。Umayalpuram K Sivaraman博士T Ramasami和Dr. Naresh，这是一本科学论文，探索了古印度古典打击乐器Mridangam的独特音调特性。这篇评论旨在弥合Mridangam的音乐卓越的主要目标受众之间的差距-听众，艺术家和制造商-和原始论文的科学严谨性，首先介绍音乐分析的概念，然后介绍和解释在此背景下所做的发现。本综述的前三章介绍了Mridangam的音乐卓越中使用的基本科学概念，并提供了以前对这种乐器的科学研究背景，从博士的开创性工作开始。CV拉曼。这也包括对《Mridangam的音乐卓越》中相应章节的简要讨论。接下来的章节都服务于解释主要的科学成果提出的音乐卓越的Mridangam在每一个相应的章节在论文中，并最终总结了相关的工作。摘要：This is an introductory review of Musical Excellence of Mridangam by Dr. Umayalpuram K Sivaraman, Dr. T Ramasami and Dr. Naresh, which is a scientific treatise exploring the unique tonal properties of the ancient Indian classical percussive instrument -- the Mridangam. This review aims to bridge the gap between the primary intended audience of Musical Excellence of Mridangam - listeners, artistes and makers -- and the scientific rigour with which the original treatise is written, by first introducing the concepts of musical analysis and then presenting and explaining the discoveries made within this context. The first three chapters of this review introduce the basic scientific concepts used in Musical Excellence of Mridangam and provides background to previous scientific research into this instrument, starting from the seminal work of Dr. CV Raman. This also includes brief discussions of the corresponding chapters in Musical Excellence of Mridangam. The next chapters all serve the purpose of explaining the main scientific results presented in Musical Excellence of Mridangam in each of the corresponding chapters in the treatise, and finally summarizing the relevance of the work.
【6】 Adapting an ASR Foundation Model for Spoken Language Assessment标题：采用ASR基础模式进行口语评估链接：https://arxiv.org/abs/2307.09378
作者：Rao Ma,Mengjie Qian,Mark J. F. Gales,Kate M. Knill机构：ALTA Institute, Machine Intelligence Lab, Department of Engineering, Cambridge University, UK备注：Submitted to SLaTE 2023摘要：准确可靠的口语评估系统的关键部分是底层的ASR模型。最近，大规模的预训练ASR基础模型，如Whisper已经可用。由于这些模型的输出被设计为人类可读，因此添加了标点符号，数字以阿拉伯数字形式呈现，并包括缩写。此外，这些模型有一种倾向，跳过输出中的不流利和犹豫。虽然这些属性有助于可读性，但对于评估候选人的能力和提供反馈没有帮助。这里需要准确地记录候选人所说的内容。在本文中，我们对Whisper输出进行了详细分析，并提出了两种解决方案：微调和软提示调谐。实验进行了公共演讲语料库和英语学习者数据集。结果表明，我们可以有效地改变Whisper的解码行为，以生成确切的话说的响应。摘要：A crucial part of an accurate and reliable spoken language assessment system is the underlying ASR model. Recently, large-scale pre-trained ASR foundation models such as Whisper have been made available. As the output of these models is designed to be human readable, punctuation is added, numbers are presented in Arabic numeric form and abbreviations are included. Additionally, these models have a tendency to skip disfluencies and hesitations in the output. Though useful for readability, these attributes are not helpful for assessing the ability of a candidate and providing feedback. Here a precise transcription of what a candidate said is needed. In this paper, we give a detailed analysis of Whisper outputs and propose two solutions: fine-tuning and soft prompt tuning. Experiments are conducted on both public speech corpora and an English learner dataset. Results show that we can effectively alter the decoding behaviour of Whisper to generate the exact words spoken in the response.
【7】 FlexiAST: Flexibility is What AST Needs标题：FlexiAST：灵活性是AST所需要的链接：https://arxiv.org/abs/2307.09286
作者：Jiu Feng,Mehmet Hamza Erol,Joon Son Chung,Arda Senocak机构：Korea Advanced Institute of Science and Technology, South Korea备注：Interspeech 2023摘要：这项工作的目的是给补丁大小的灵活性，音频频谱图Transformers（AST）。AST的最新进展已经在各种基于音频的任务中显示出优异的性能。然而，标准的AST的性能急剧下降时，使用不同的补丁大小从训练过程中使用的评估。因此，AST模型通常被重新训练以适应补丁大小的变化。为了克服这一限制，本文提出了一个训练过程，以提供灵活性，标准AST模型，而无需架构的变化，使他们能够在推理阶段的各种补丁大小- FlexiAST。这种训练方法简单地利用随机补丁大小选择和补丁和位置嵌入权重的大小调整。我们的实验表明，FlexiAST提供了类似的性能标准AST模型，同时保持其评估能力，在不同的补丁大小在不同的数据集上的音频分类任务。摘要：The objective of this work is to give patch-size flexibility to Audio Spectrogram Transformers (AST). Recent advancements in ASTs have shown superior performance in various audio-based tasks. However, the performance of standard ASTs degrades drastically when evaluated using different patch sizes from that used during training. As a result, AST models are typically re-trained to accommodate changes in patch sizes. To overcome this limitation, this paper proposes a training procedure to provide flexibility to standard AST models without architectural changes, allowing them to work with various patch sizes at the inference stage - FlexiAST. This proposed training approach simply utilizes random patch size selection and resizing of patch and positional embedding weights. Our experiments show that FlexiAST gives similar performance to standard AST models while maintaining its evaluation ability at various patch sizes on different datasets for audio classification tasks.
【8】 Detecting Throat Cancer from Speech Signals Using Machine Learning: A  Reproducible Literature Review标题：利用机器学习从语音信号中检测喉癌：一篇可重复的文献综述链接：https://arxiv.org/abs/2307.09230
作者：Mary Paterson,James Moor,Luisa Cutillo机构：University of Leeds, Leeds Teaching Hospitals NHS Trust备注：19 pages, 10 figures摘要：在这项工作中，我们对使用机器学习和人工智能从语音记录中检测喉癌的现有文献进行了范围审查。我们发现在这一领域的22篇论文，并讨论了他们的方法和结果。我们将这些文件分成两组-九个执行二进制分类，和13个执行多类分类。这些论文提出了一系列最常用的神经网络方法。在分类之前，还从音频中提取了许多特征，其中最常见的是梅尔频率倒谱系数。本次检索中发现的论文均没有相关的代码存储库，因此不可复制。因此，我们创建了一个公开可用的代码库我们自己的分类器。我们使用迁移学习的多类问题，分类三个病理和健康对照。使用这种技术，我们实现了53.54%的未加权平均召回率，83.14%的敏感性，64.00%的特异性。我们比较我们的分类器与相同的数据集上获得的结果，并发现类似的结果。摘要：In this work we perform a scoping review of the current literature on the detection of throat cancer from speech recordings using machine learning and artificial intelligence. We find 22 papers within this area and discuss their methods and results. We split these papers into two groups - nine performing binary classification, and 13 performing multi-class classification. The papers present a range of methods with neural networks being most commonly implemented. Many features are also extracted from the audio before classification, with the most common bring mel-frequency cepstral coefficients. None of the papers found in this search have associated code repositories and as such are not reproducible. Therefore, we create a publicly available code repository of our own classifiers. We use transfer learning on a multi-class problem, classifying three pathologies and healthy controls. Using this technique we achieve an unweighted average recall of 53.54%, sensitivity of 83.14%, and specificity of 64.00%. We compare our classifiers with the results obtained on the same dataset and find similar results.
【9】 OxfordVGG Submission to the EGO4D AV Transcription Challenge标题：OxfordVGG提交给EGO4D AV转录挑战赛链接：https://arxiv.org/abs/2307.09006
作者：Jaesung Huh,Max Bain,Andrew Zisserman机构：Visual Geometry Group, University of Oxford, UK备注：Technical Report摘要：本报告介绍了我们提交的牛津VGG团队EGO4D视听（AV）自动语音识别挑战赛2023的技术细节。我们提出了WhisperX，一个系统，高效的语音转录的长格式音频与字级时间对齐，以及两个文本规范这是公开可用的。我们最终提交的作品在挑战测试集上获得了56.0%的单词错误率（WER），在排行榜上排名第一。所有基准代码和模型均可在https://github.com/m-bain/whisperX上获得。摘要：This report presents the technical details of our submission on the EGO4D Audio-Visual (AV) Automatic Speech Recognition Challenge 2023 from the OxfordVGG team. We present WhisperX, a system for efficient speech transcription of long-form audio with word-level time alignment, along with two text normalisers which are publicly available. Our final submission obtained 56.0% of the Word Error Rate (WER) on the challenge test set, ranked 1st on the leaderboard. All baseline codes and models are available on https://github.com/m-bain/whisperX.
【10】 On Computing In the Network: Covid-19 Coughs Detection Case Study标题：论网络计算：新冠肺炎咳嗽检测案例研究链接：https://arxiv.org/abs/2307.08902
作者：Soukaina Ouledsidi Ali,Zakaria Ait Hmitti,Halima Elbiaze,Roch Glitho机构：Glitho,  Universit´e du Qu´ebec A Montr´eal, Montreal, Canada,  Concordia University, Montreal, Canada备注：None摘要：网络中的计算（COIN）是一种有前途的技术，其允许在诸如交换机和网络接口卡之类的网络设备内执行处理。时间敏感型应用可以通过在云-边缘-雾连续体中灵活地分配缓存和计算任务来实现其服务质量（QoS）目标。本文强调了网络计算与边缘计算相比在延迟和流量过滤方面的优势。我们考虑了一个与机场环境中的Covid-19警报应用相关的关键用例。通过咳嗽分析监测到达的旅客，以便发现潜在的感染病例并进行隔离进行医学检查。已经在使用网络内计算的架构和使用边缘计算的架构之间进行了性能比较。我们使用模拟表明，在往返时间（RTT）和流量过滤方面，网络内计算优于边缘计算。摘要：Computing in the network (COIN) is a promising technology that allows processing to be carried out within network devices such as switches and network interface cards. Time sensitive application can achieve their quality of service (QoS) target by flexibly distributing the caching and computing tasks in the cloud-edge-mist continuum. This paper highlights the advantages of in-network computing, comparing to edge computing, in terms of latency and traffic filtering. We consider a critical use case related to Covid-19 alert application in an airport setting. Arriving travelers are monitored through cough analysis so that potentially infected cases can be detected and isolated for medical tests. A performance comparison has been done between an architecture using in-network computing and another one using edge computing. We show using simulations that in-network computing outperforms edge computing in terms of Round Trip Time (RTT) and traffic filtering.
机器翻译，仅供参考