今日论文合集：cs.SD语音5篇，eess.AS音频处理5篇。
cs.SD语音

【1】 Detecting human and non-human vocal productions in large scale audio  recordings

标题：在大规模音频记录中检测人类和非人类发声产物

链接：https://arxiv.org/abs/2302.07640

作者：Guillem Bonafos,Pierre Pudlo,Jean-Marc Freyermuth,Thierry Legou,Joël Fagot,Samuel Tronçon,Arnaud Rey
机构：Aix Marseille Univ, CNRS, I,M, Marseille, France, Aix Marseille Univ, CNRS, LPC, Marseille, France, Aix Marseille Univ, CNRS, LPL, Aix-en-Provence, France, Aix Marseille Univ, ILCB, Aix-en-Provence, France, R´esurgences R&D, Arles, France
摘要：我们提出了一个自动数据处理流水线来从大规模的自然录音中提取声乐作品。通过一系列计算步骤（开窗、创建噪声类、数据扩增、重新采样、迁移学习、贝叶斯优化），它自动训练神经网络，用于检测噪声数据流中的各种类型的自然发声，而不需要标记数据的大样本。我们在两个不同的数据集上进行了测试，一个来自灵长类动物研究中心记录的一群几内亚狒狒，另一个来自家中记录的人类婴儿。该管道在72分钟和77分钟的标记音频记录上训练模型，准确率分别为94.58%和99.76%。然后，它被用于处理443和174小时的自然连续记录，并且它创建了分别为38.8和35.2小时的两个新数据库。我们讨论了这种方法的优点和局限性，它可以应用于任何大规模的音频记录。
摘要：We propose an automatic data processing pipeline to extract vocal productions from large-scale natural audio recordings. Through a series of computational steps (windowing, creation of a noise class, data augmentation, re-sampling, transfer learning, Bayesian optimisation), it automatically trains a neural network for detecting various types of natural vocal productions in a noisy data stream without requiring a large sample of labeled data. We test it on two different data sets, one from a group of Guinea baboons recorded from a primate research center and one from human babies recorded at home. The pipeline trains a model on 72 and 77 minutes of labeled audio recordings, with an accuracy of 94.58% and 99.76%. It is then used to process 443 and 174 hours of natural continuous recordings and it creates two new databases of 38.8 and 35.2 hours, respectively. We discuss the strengths and limitations of this approach that can be applied to any massive audio recording.

【2】 Unsupervised classification to improve the quality of a bird song  recording dataset

标题：改进鸟鸣记录数据集质量的非监督分类方法

链接：https://arxiv.org/abs/2302.07560

作者：Félix Michaud,Jérôme Sueur,Maxime Le Cesne,Sylvain Haupert
机构：Institut Systématique, Évolution, Biodiversité (ISYEB), Muséum national d’Histoire, naturelle, CNRS, Sorbonne Université, EPHE, Université des Antilles, rue Cuvier, Paris, France
备注：None
摘要：Xeno-Canto等开放音频数据库被广泛用于构建数据集，以探索鸟鸣曲目或通过深度学习算法训练鸟鸣自动分类模型。然而，这样的数据库遭受鸟的声音被弱标记的事实：物种名称被归属于没有时间戳的每个音频记录，所述时间戳提供感兴趣的鸟鸣的时间定位。手动注释可以解决这个问题，但是它们非常耗时、依赖于专家，并且不能在大型数据集上运行。另一种解决方案在于使用标记功能，该标记功能在向每个分段的音频样本分配标签之前自动地分段音频记录。虽然标签函数被引入以加速强标签分配，但是它们的分类性能仍然是未知的。为了解决这一问题并减少大型鸟鸣数据集中的标签噪声（错误的标签分配），我们引入了一种以数据为中心的新型标签函数，该函数由三个连续步骤组成：1）时间-频率声音单元分割，2）针对每个声音单元的特征计算，以及3）使用无监督DBSCAN算法或监督BirdNET神经网络将每个声音单元分类为鸟鸣或噪声。对标记函数进行了优化、验证，并在44种西古北常见鸟类的鸣唱上进行了测试。我们首先展示了单独的鸟鸣分割聚集了10%到83%的标签噪声，这取决于物种。我们还证明了我们的标记函数能够显著减少数据集中存在的初始标记噪声，最多减少三分之一。最后，我们讨论了不同的机会来设计合适的标记函数，以最小的专家注释工作来构建高质量的动物发声。
摘要：Open audio databases such as Xeno-Canto are widely used to build datasets to explore bird song repertoire or to train models for automatic bird sound classification by deep learning algorithms. However, such databases suffer from the fact that bird sounds are weakly labelled: a species name is attributed to each audio recording without timestamps that provide the temporal localization of the bird song of interest. Manual annotations can solve this issue, but they are time consuming, expert-dependent, and cannot run on large datasets. Another solution consists in using a labelling function that automatically segments audio recordings before assigning a label to each segmented audio sample. Although labelling functions were introduced to expedite strong label assignment, their classification performance remains mostly unknown. To address this issue and reduce label noise (wrong label assignment) in large bird song datasets, we introduce a data-centric novel labelling function composed of three successive steps: 1) time-frequency sound unit segmentation, 2) feature computation for each sound unit, and 3) classification of each sound unit as bird song or noise with either an unsupervised DBSCAN algorithm or the supervised BirdNET neural network. The labelling function was optimized, validated, and tested on the songs of 44 West-Palearctic common bird species. We first showed that the segmentation of bird songs alone aggregated from 10% to 83% of label noise depending on the species. We also demonstrated that our labelling function was able to significantly reduce the initial label noise present in the dataset by up to a factor of three. Finally, we discuss different opportunities to design suitable labelling functions to build high-quality animal vocalizations with minimum expert annotation effort.

【3】 Fast Blind Audio Copy-Move Detection and Localization Using Local  Feature Tensors in Noise

标题：噪声中基于局部特征张量的音频复制移动快速盲检测与定位

链接：https://arxiv.org/abs/2302.07584

作者：Dong Yang,Mingle Liu,Muyong Cao
机构：GVoice Team, Tencent
摘要：改变数字音频的音频编辑软件的日益增加的可用性及其易用性允许以低成本创建伪造。拷贝-移动伪造（copy-move forgery，CMF）是一种最简单、最流行的音频伪造，它是通过复制和粘贴同一音频中的音频片段并对其进行后处理而产生的。样本/帧比较、声学特征相干性搜索和动态时间规整。但是这些方法将遭受计算复杂性和/或对噪声和后处理敏感。本文提出了一种新的基于局部特征张量的复制移动检测算法，该算法可以应用于一个特殊的局部敏感的类哈希过程的变换副本检测和定位问题。在大规模在线实时音频数据集上的实验结果表明，即使在短至小数秒的伪造语音片段上，该方法也能有效地检测和定位复制-移动伪造.该方法计算效率高，对经过重采样、滤波、抖动、压缩、剪切等严重非线性变换处理的音频信号，甚至对被背景噪声和音乐污染的音频信号都具有很强的鲁棒性。因此，所提出的技术提供了一种有效且可靠的拷贝-移动伪造检测方式，其增加了音频在实际取证应用中的可信度
摘要：The increasing availability of audio editing software altering digital audios and their ease of use allows create forgeries at low cost. A copy-move forgery (CMF) is one of easiest and popular audio forgeries, which created by copying and pasting audio segments within the same audio, and potentially post-processing it. Three main approaches to audio copy-move detection exist nowadays: samples/frames comparison, acoustic features coherence searching and dynamic time warping. But these approaches will suffer from computational complexity and/or sensitive to noise and post-processing. In this paper, we propose a new local feature tensors-based copy-move detection algorithm that can be applied to transformed duplicates detection and localization problem to a special locality sensitive hash like procedure. The experimental results with massive online real-time audios datasets reveal that the proposed technique effectively determines and locating copy-move forgeries even on a forged speech segment are as short as fractional second. This method is also computational efficient and robust against the audios processed with severe nonlinear transformation, such as resampling, filtering, jsittering, compression and cropping, even contaminated with background noise and music. Hence, the proposed technique provides an efficient and reliable way of copy-move forgery detection that increases the credibility of audio in practical forensics applications

【4】 Confidence Score Based Speaker Adaptation of Conformer Speech  Recognition Systems

标题：基于置信度的一致性语音识别系统说话人自适应

链接：https://arxiv.org/abs/2302.07521

作者：Jiajun Deng,Xurong Xie,Tianzi Wang,Mingyu Cui,Boyang Xue,Zengrui Jin,Guinan Li,Shujie Hu,Xunying Liu
备注：IEEE/ACM Transactions on Audio, Speech, and Language Processing
摘要：说话人自适应技术提供了一个强大的解决方案来定制自动语音识别（ASR）系统的个人用户。基于无监督模型的说话人自适应技术在数据密集型端到端ASR系统中的实际应用受到说话人级别数据的稀缺性和对转录错误的性能敏感性的阻碍。为了解决这些问题，使用一组紧凑且数据有效的说话人相关（SD）参数表示来促进现有Conformer ASR系统的说话人自适应训练和测试时无监督说话人自适应。使用对说话者级别自适应数据的较少错误子集的基于置信度分数的选择来降低对监督质量的敏感度。提出了两个轻量级的置信度估计模块，以产生更可靠的置信度。通过使用贝叶斯学习对SD参数不确定性进行建模，解决了数据稀疏性问题（因数据选择而加剧）。在基准300小时Switchboard和233小时AMI数据集上的实验表明，所提出的基于置信度分数的自适应方案始终优于基线说话人无关（SI）Conformer模型和传统的非贝叶斯、不使用说话人数据选择的基于点估计的自适应。在外部Transformer和LSTM语言模型重新评分后，保持了类似的一致性能改进。特别是，在300小时Switchboard语料库中，NIST Hub5'00、RT02和RT03评价集的基线SI一致者WER分别降低了1.0%、1.3%和1.4%（绝对值为9.5%、10.9%和11.3%），具有统计学显著性。在AMI开发和评价集也获得了相似的WER绝对降低2.7%和3.3%（相对降低8.9%和10.2%）。
摘要：Speaker adaptation techniques provide a powerful solution to customise automatic speech recognition (ASR) systems for individual users. Practical application of unsupervised model-based speaker adaptation techniques to data intensive end-to-end ASR systems is hindered by the scarcity of speaker-level data and performance sensitivity to transcription errors. To address these issues, a set of compact and data efficient speaker-dependent (SD) parameter representations are used to facilitate both speaker adaptive training and test-time unsupervised speaker adaptation of state-of-the-art Conformer ASR systems. The sensitivity to supervision quality is reduced using a confidence score-based selection of the less erroneous subset of speaker-level adaptation data. Two lightweight confidence score estimation modules are proposed to produce more reliable confidence scores. The data sparsity issue, which is exacerbated by data selection, is addressed by modelling the SD parameter uncertainty using Bayesian learning. Experiments on the benchmark 300-hour Switchboard and the 233-hour AMI datasets suggest that the proposed confidence score-based adaptation schemes consistently outperformed the baseline speaker-independent (SI) Conformer model and conventional non-Bayesian, point estimate-based adaptation using no speaker data selection. Similar consistent performance improvements were retained after external Transformer and LSTM language model rescoring. In particular, on the 300-hour Switchboard corpus, statistically significant WER reductions of 1.0%, 1.3%, and 1.4% absolute (9.5%, 10.9%, and 11.3% relative) were obtained over the baseline SI Conformer on the NIST Hub5'00, RT02, and RT03 evaluation sets respectively. Similar WER reductions of 2.7% and 3.3% absolute (8.9% and 10.2% relative) were also obtained on the AMI development and evaluation sets.

【5】 A dataset for Audio-Visual Sound Event Detection in Movies

标题：一种用于电影视听声音事件检测的数据集

链接：https://arxiv.org/abs/2302.07315

作者：Rajat Hebbar,Digbalay Bose,Krishna Somandepalli,Veena Vijai,Shrikanth Narayanan
机构：Signal Analysis and Interpretation Laboratory, University of Southern California
摘要：音频事件检测是广泛研究的音频处理任务，其应用范围从自动驾驶汽车到医疗保健。Audioset等野外数据集推动了这一领域的研究。然而，许多努力通常涉及手动注释和验证，这在大规模执行时是昂贵的。电影描述了各种现实生活和虚构的场景，这使得它们成为挖掘广泛音频事件的丰富资源。在这项工作中，我们提出了一个音频事件的数据集称为字幕对齐电影声音（SAM-S）。我们使用公开可用的隐藏字幕脚本从430部电影中自动挖掘超过11万个音频事件。我们识别三个维度来分类音频事件：声音、来源、质量，并呈现生成245种声音的最终分类所涉及的步骤。我们讨论了生成分类所涉及的选择，并强调了数据集中声音以人为中心的本质。我们为纯音频声音分类建立了平均精度为34.76%的基线性能，并表明结合视觉信息可以进一步提高约5%的性能。可在www.example.com上获取数据和代码以供研究https://github.com/usc-sail/mica-subtitle-aligned-movie-sounds
摘要：Audio event detection is a widely studied audio processing task, with applications ranging from self-driving cars to healthcare. In-the-wild datasets such as Audioset have propelled research in this field. However, many efforts typically involve manual annotation and verification, which is expensive to perform at scale. Movies depict various real-life and fictional scenarios which makes them a rich resource for mining a wide-range of audio events. In this work, we present a dataset of audio events called Subtitle-Aligned Movie Sounds (SAM-S). We use publicly-available closed-caption transcripts to automatically mine over 110K audio events from 430 movies. We identify three dimensions to categorize audio events: sound, source, quality, and present the steps involved to produce a final taxonomy of 245 sounds. We discuss the choices involved in generating the taxonomy, and also highlight the human-centered nature of sounds in our dataset. We establish a baseline performance for audio-only sound classification of 34.76% mean average precision and show that incorporating visual information can further improve the performance by about 5%. Data and code are made available for research at https://github.com/usc-sail/mica-subtitle-aligned-movie-sounds
eess.AS音频处理

【1】 Fast Blind Audio Copy-Move Detection and Localization Using Local  Feature Tensors in Noise

标题：噪声中基于局部特征张量的音频复制移动快速盲检测与定位

链接：https://arxiv.org/abs/2302.07584

*与cs.SD语音【3】为同一篇

作者：Dong Yang,Mingle Liu,Muyong Cao
机构：GVoice Team, Tencent
摘要：改变数字音频的音频编辑软件的日益增加的可用性及其易用性允许以低成本创建伪造。拷贝-移动伪造（copy-move forgery，CMF）是一种最简单、最流行的音频伪造，它是通过复制和粘贴同一音频中的音频片段并对其进行后处理而产生的。样本/帧比较、声学特征相干性搜索和动态时间规整。但是这些方法将遭受计算复杂性和/或对噪声和后处理敏感。本文提出了一种新的基于局部特征张量的复制移动检测算法，该算法可以应用于一个特殊的局部敏感的类哈希过程的变换副本检测和定位问题。在大规模在线实时音频数据集上的实验结果表明，即使在短至小数秒的伪造语音片段上，该方法也能有效地检测和定位复制-移动伪造.该方法计算效率高，对经过重采样、滤波、抖动、压缩、剪切等严重非线性变换处理的音频信号，甚至对被背景噪声和音乐污染的音频信号都具有很强的鲁棒性。因此，所提出的技术提供了一种有效且可靠的拷贝-移动伪造检测方式，其增加了音频在实际取证应用中的可信度
摘要：The increasing availability of audio editing software altering digital audios and their ease of use allows create forgeries at low cost. A copy-move forgery (CMF) is one of easiest and popular audio forgeries, which created by copying and pasting audio segments within the same audio, and potentially post-processing it. Three main approaches to audio copy-move detection exist nowadays: samples/frames comparison, acoustic features coherence searching and dynamic time warping. But these approaches will suffer from computational complexity and/or sensitive to noise and post-processing. In this paper, we propose a new local feature tensors-based copy-move detection algorithm that can be applied to transformed duplicates detection and localization problem to a special locality sensitive hash like procedure. The experimental results with massive online real-time audios datasets reveal that the proposed technique effectively determines and locating copy-move forgeries even on a forged speech segment are as short as fractional second. This method is also computational efficient and robust against the audios processed with severe nonlinear transformation, such as resampling, filtering, jsittering, compression and cropping, even contaminated with background noise and music. Hence, the proposed technique provides an efficient and reliable way of copy-move forgery detection that increases the credibility of audio in practical forensics applications

【2】 Confidence Score Based Speaker Adaptation of Conformer Speech  Recognition Systems

标题：基于置信度的一致性语音识别系统说话人自适应

链接：https://arxiv.org/abs/2302.07521

*与cs.SD语音【4】为同一篇

作者：Jiajun Deng,Xurong Xie,Tianzi Wang,Mingyu Cui,Boyang Xue,Zengrui Jin,Guinan Li,Shujie Hu,Xunying Liu
备注：IEEE/ACM Transactions on Audio, Speech, and Language Processing
摘要：说话人自适应技术提供了一个强大的解决方案来定制自动语音识别（ASR）系统的个人用户。基于无监督模型的说话人自适应技术在数据密集型端到端ASR系统中的实际应用受到说话人级别数据的稀缺性和对转录错误的性能敏感性的阻碍。为了解决这些问题，使用一组紧凑且数据有效的说话人相关（SD）参数表示来促进现有Conformer ASR系统的说话人自适应训练和测试时无监督说话人自适应。使用对说话者级别自适应数据的较少错误子集的基于置信度分数的选择来降低对监督质量的敏感度。提出了两个轻量级的置信度估计模块，以产生更可靠的置信度。通过使用贝叶斯学习对SD参数不确定性进行建模，解决了数据稀疏性问题（因数据选择而加剧）。在基准300小时Switchboard和233小时AMI数据集上的实验表明，所提出的基于置信度分数的自适应方案始终优于基线说话人无关（SI）Conformer模型和传统的非贝叶斯、不使用说话人数据选择的基于点估计的自适应。在外部Transformer和LSTM语言模型重新评分后，保持了类似的一致性能改进。特别是，在300小时Switchboard语料库中，NIST Hub5'00、RT02和RT03评价集的基线SI一致者WER分别降低了1.0%、1.3%和1.4%（绝对值为9.5%、10.9%和11.3%），具有统计学显著性。在AMI开发和评价集也获得了相似的WER绝对降低2.7%和3.3%（相对降低8.9%和10.2%）。
摘要：Speaker adaptation techniques provide a powerful solution to customise automatic speech recognition (ASR) systems for individual users. Practical application of unsupervised model-based speaker adaptation techniques to data intensive end-to-end ASR systems is hindered by the scarcity of speaker-level data and performance sensitivity to transcription errors. To address these issues, a set of compact and data efficient speaker-dependent (SD) parameter representations are used to facilitate both speaker adaptive training and test-time unsupervised speaker adaptation of state-of-the-art Conformer ASR systems. The sensitivity to supervision quality is reduced using a confidence score-based selection of the less erroneous subset of speaker-level adaptation data. Two lightweight confidence score estimation modules are proposed to produce more reliable confidence scores. The data sparsity issue, which is exacerbated by data selection, is addressed by modelling the SD parameter uncertainty using Bayesian learning. Experiments on the benchmark 300-hour Switchboard and the 233-hour AMI datasets suggest that the proposed confidence score-based adaptation schemes consistently outperformed the baseline speaker-independent (SI) Conformer model and conventional non-Bayesian, point estimate-based adaptation using no speaker data selection. Similar consistent performance improvements were retained after external Transformer and LSTM language model rescoring. In particular, on the 300-hour Switchboard corpus, statistically significant WER reductions of 1.0%, 1.3%, and 1.4% absolute (9.5%, 10.9%, and 11.3% relative) were obtained over the baseline SI Conformer on the NIST Hub5'00, RT02, and RT03 evaluation sets respectively. Similar WER reductions of 2.7% and 3.3% absolute (8.9% and 10.2% relative) were also obtained on the AMI development and evaluation sets.

【3】 A dataset for Audio-Visual Sound Event Detection in Movies

标题：一种用于电影视听声音事件检测的数据集

链接：https://arxiv.org/abs/2302.07315

*与cs.SD语音【5】为同一篇

作者：Rajat Hebbar,Digbalay Bose,Krishna Somandepalli,Veena Vijai,Shrikanth Narayanan
机构：Signal Analysis and Interpretation Laboratory, University of Southern California
摘要：音频事件检测是广泛研究的音频处理任务，其应用范围从自动驾驶汽车到医疗保健。Audioset等野外数据集推动了这一领域的研究。然而，许多努力通常涉及手动注释和验证，这在大规模执行时是昂贵的。电影描述了各种现实生活和虚构的场景，这使得它们成为挖掘广泛音频事件的丰富资源。在这项工作中，我们提出了一个音频事件的数据集称为字幕对齐电影声音（SAM-S）。我们使用公开可用的隐藏字幕脚本从430部电影中自动挖掘超过11万个音频事件。我们识别三个维度来分类音频事件：声音、来源、质量，并呈现生成245种声音的最终分类所涉及的步骤。我们讨论了生成分类所涉及的选择，并强调了数据集中声音以人为中心的本质。我们为纯音频声音分类建立了平均精度为34.76%的基线性能，并表明结合视觉信息可以进一步提高约5%的性能。可在www.example.com上获取数据和代码以供研究https://github.com/usc-sail/mica-subtitle-aligned-movie-sounds
摘要：Audio event detection is a widely studied audio processing task, with applications ranging from self-driving cars to healthcare. In-the-wild datasets such as Audioset have propelled research in this field. However, many efforts typically involve manual annotation and verification, which is expensive to perform at scale. Movies depict various real-life and fictional scenarios which makes them a rich resource for mining a wide-range of audio events. In this work, we present a dataset of audio events called Subtitle-Aligned Movie Sounds (SAM-S). We use publicly-available closed-caption transcripts to automatically mine over 110K audio events from 430 movies. We identify three dimensions to categorize audio events: sound, source, quality, and present the steps involved to produce a final taxonomy of 245 sounds. We discuss the choices involved in generating the taxonomy, and also highlight the human-centered nature of sounds in our dataset. We establish a baseline performance for audio-only sound classification of 34.76% mean average precision and show that incorporating visual information can further improve the performance by about 5%. Data and code are made available for research at https://github.com/usc-sail/mica-subtitle-aligned-movie-sounds

【4】 Detecting human and non-human vocal productions in large scale audio  recordings

标题：在大规模音频记录中检测人类和非人类发声产物

链接：https://arxiv.org/abs/2302.07640

*与cs.SD语音【1】为同一篇

作者：Guillem Bonafos,Pierre Pudlo,Jean-Marc Freyermuth,Thierry Legou,Joël Fagot,Samuel Tronçon,Arnaud Rey
机构：Aix Marseille Univ, CNRS, I,M, Marseille, France, Aix Marseille Univ, CNRS, LPC, Marseille, France, Aix Marseille Univ, CNRS, LPL, Aix-en-Provence, France, Aix Marseille Univ, ILCB, Aix-en-Provence, France, R´esurgences R&D, Arles, France
摘要：我们提出了一个自动数据处理流水线来从大规模的自然录音中提取声乐作品。通过一系列计算步骤（开窗、创建噪声类、数据扩增、重新采样、迁移学习、贝叶斯优化），它自动训练神经网络，用于检测噪声数据流中的各种类型的自然发声，而不需要标记数据的大样本。我们在两个不同的数据集上进行了测试，一个来自灵长类动物研究中心记录的一群几内亚狒狒，另一个来自家中记录的人类婴儿。该管道在72分钟和77分钟的标记音频记录上训练模型，准确率分别为94.58%和99.76%。然后，它被用于处理443和174小时的自然连续记录，并且它创建了分别为38.8和35.2小时的两个新数据库。我们讨论了这种方法的优点和局限性，它可以应用于任何大规模的音频记录。
摘要：We propose an automatic data processing pipeline to extract vocal productions from large-scale natural audio recordings. Through a series of computational steps (windowing, creation of a noise class, data augmentation, re-sampling, transfer learning, Bayesian optimisation), it automatically trains a neural network for detecting various types of natural vocal productions in a noisy data stream without requiring a large sample of labeled data. We test it on two different data sets, one from a group of Guinea baboons recorded from a primate research center and one from human babies recorded at home. The pipeline trains a model on 72 and 77 minutes of labeled audio recordings, with an accuracy of 94.58% and 99.76%. It is then used to process 443 and 174 hours of natural continuous recordings and it creates two new databases of 38.8 and 35.2 hours, respectively. We discuss the strengths and limitations of this approach that can be applied to any massive audio recording.

【5】 Unsupervised classification to improve the quality of a bird song  recording dataset

标题：改进鸟鸣记录数据集质量的非监督分类方法

链接：https://arxiv.org/abs/2302.07560

*与cs.SD语音【2】为同一篇

作者：Félix Michaud,Jérôme Sueur,Maxime Le Cesne,Sylvain Haupert
机构：Institut Systématique, Évolution, Biodiversité (ISYEB), Muséum national d’Histoire, naturelle, CNRS, Sorbonne Université, EPHE, Université des Antilles, rue Cuvier, Paris, France
备注：None
摘要：Xeno-Canto等开放音频数据库被广泛用于构建数据集，以探索鸟鸣曲目或通过深度学习算法训练鸟鸣自动分类模型。然而，这样的数据库遭受鸟的声音被弱标记的事实：物种名称被归属于没有时间戳的每个音频记录，所述时间戳提供感兴趣的鸟鸣的时间定位。手动注释可以解决这个问题，但是它们非常耗时、依赖于专家，并且不能在大型数据集上运行。另一种解决方案在于使用标记功能，该标记功能在向每个分段的音频样本分配标签之前自动地分段音频记录。虽然标签函数被引入以加速强标签分配，但是它们的分类性能仍然是未知的。为了解决这一问题并减少大型鸟鸣数据集中的标签噪声（错误的标签分配），我们引入了一种以数据为中心的新型标签函数，该函数由三个连续步骤组成：1）时间-频率声音单元分割，2）针对每个声音单元的特征计算，以及3）使用无监督DBSCAN算法或监督BirdNET神经网络将每个声音单元分类为鸟鸣或噪声。对标记函数进行了优化、验证，并在44种西古北常见鸟类的鸣唱上进行了测试。我们首先展示了单独的鸟鸣分割聚集了10%到83%的标签噪声，这取决于物种。我们还证明了我们的标记函数能够显著减少数据集中存在的初始标记噪声，最多减少三分之一。最后，我们讨论了不同的机会来设计合适的标记函数，以最小的专家注释工作来构建高质量的动物发声。
摘要：Open audio databases such as Xeno-Canto are widely used to build datasets to explore bird song repertoire or to train models for automatic bird sound classification by deep learning algorithms. However, such databases suffer from the fact that bird sounds are weakly labelled: a species name is attributed to each audio recording without timestamps that provide the temporal localization of the bird song of interest. Manual annotations can solve this issue, but they are time consuming, expert-dependent, and cannot run on large datasets. Another solution consists in using a labelling function that automatically segments audio recordings before assigning a label to each segmented audio sample. Although labelling functions were introduced to expedite strong label assignment, their classification performance remains mostly unknown. To address this issue and reduce label noise (wrong label assignment) in large bird song datasets, we introduce a data-centric novel labelling function composed of three successive steps: 1) time-frequency sound unit segmentation, 2) feature computation for each sound unit, and 3) classification of each sound unit as bird song or noise with either an unsupervised DBSCAN algorithm or the supervised BirdNET neural network. The labelling function was optimized, validated, and tested on the songs of 44 West-Palearctic common bird species. We first showed that the segmentation of bird songs alone aggregated from 10% to 83% of label noise depending on the species. We also demonstrated that our labelling function was able to significantly reduce the initial label noise present in the dataset by up to a factor of three. Finally, we discuss different opportunities to design suitable labelling functions to build high-quality animal vocalizations with minimum expert annotation effort.

机器翻译，仅供参考