今日论文合集：cs.SD语音9篇，eess.AS音频处理9篇。
cs.SD语音

【1】 Enhancing Speech-to-Speech Translation with Multiple TTS Targets标题：利用多个TTS目标增强语音到语音的翻译链接：http://arxiv.org/pdf/2304.04618v1

作者：Jiatong Shi,Yun Tang,Ann Lee,Hirofumi Inaguma,Changhan Wang,Juan Pino,Shinji Watanabe机构：Carnegie Mellon University, Meta AI摘要：人们已经知道，直接语音到语音翻译（S2 ST）模型通常遭受的数据稀缺的问题，因为有限的现有的平行材料的源和目标语音。因此，为了训练直接S2 ST系统，以前的工作通常利用文本到语音（TTS）系统通过增强来自语音到文本翻译（S2 TT）的数据来生成目标语言的样本。然而，有一个有限的调查如何合成的目标语音会影响S2 ST模型。在这项工作中，我们分析了直接S2 ST模型的合成目标语音的变化的效果。我们发现，简单地结合不同的TTS系统的目标语音可以潜在地提高S2 ST性能。接下来，我们还提出了一个多任务框架，共同优化S2 ST系统与来自不同的TTS系统的多个目标。大量的实验表明，我们提出的框架实现了一致的改进（2.8 BLEU）在Fisher西班牙语-英语数据集的基线。摘要：It has been known that direct speech-to-speech translation (S2ST) models usually suffer from the data scarcity issue because of the limited existing parallel materials for both source and target speech. Therefore to train a direct S2ST system, previous works usually utilize text-to-speech (TTS) systems to generate samples in the target language by augmenting the data from speech-to-text translation (S2TT). However, there is a limited investigation into how the synthesized target speech would affect the S2ST models. In this work, we analyze the effect of changing synthesized target speech for direct S2ST models. We find that simply combining the target speech from different TTS systems can potentially improve the S2ST performances. Following that, we also propose a multi-task framework that jointly optimizes the S2ST system with multiple targets from different TTS systems. Extensive experiments demonstrate that our proposed framework achieves consistent improvements (2.8 BLEU) over the baselines on the Fisher Spanish-English dataset.

【2】 In-situ crack and keyhole pore detection in laser directed energy deposition through acoustic signal and deep learning标题：基于声信号和深度学习的激光定向能沉积裂纹和小孔原位检测链接：http://arxiv.org/pdf/2304.04598v1

作者：Lequn Chen,Xiling Yao,Chaolin Tan,Weiyang He,Jinlong Su,Fei Weng,Youxiang Chew,Nicholas Poh Huat Ng,Seung Ki Moon机构：a Advanced Remanufacturing and Technology Centre (ARTC), ASTAR, Cleantech Loop, Singapore, b School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore备注：36 Pages, 16 Figures, accepted at journal Additive Manufacturing摘要：裂纹和小孔是激光定向能量沉积（LDED）制备的合金中的有害缺陷。激光-材料相互作用声音可以保持关于潜在的复杂物理事件（诸如裂纹扩展和孔隙形成）的信息。然而，由于嘈杂的环境和复杂的信号内容，在LDED基于声学的监测很少受到关注。本文提出了一种新的基于声学的LDED原位缺陷检测策略。本研究的主要贡献是开发一种原位声学信号去噪、特征提取和声音分类管道，该管道采用卷积神经网络（CNN）进行在线缺陷预测。显微镜图像用于识别部件内的裂纹和锁眼孔的位置。缺陷位置与声学信号在空间上配准。各种声学特征对应的无缺陷的区域，裂纹，和小孔孔隙提取和分析，在时域，频域，和时间-频率表示。CNN模型进行训练，以预测缺陷的发生，使用梅尔频率倒谱系数（MFCC）的lasermaterial相互作用的声音。将CNN模型与在去噪声学数据集和原始声学数据集上训练的各种经典机器学习模型进行比较。验证结果表明，在去噪数据集上训练的CNN模型优于其他模型，具有最高的总体准确度（89%），小孔预测准确度（93%）和AUC-ROC评分（98%）。此外，经过训练的CNN模型可以部署到内部开发的软件平台中，用于在线质量监控。所提出的策略是第一次使用具有深度学习的声学信号在LDED过程中进行原位缺陷检测的研究。摘要：Cracks and keyhole pores are detrimental defects in alloys produced by laser directed energy deposition (LDED). Laser-material interaction sound may hold information about underlying complex physical events such as crack propagation and pores formation. However, due to the noisy environment and intricate signal content, acoustic-based monitoring in LDED has received little attention. This paper proposes a novel acoustic-based in-situ defect detection strategy in LDED. The key contribution of this study is to develop an in-situ acoustic signal denoising, feature extraction, and sound classification pipeline that incorporates convolutional neural networks (CNN) for online defect prediction. Microscope images are used to identify locations of the cracks and keyhole pores within a part. The defect locations are spatiotemporally registered with acoustic signal. Various acoustic features corresponding to defect-free regions, cracks, and keyhole pores are extracted and analysed in time-domain, frequency-domain, and time-frequency representations. The CNN model is trained to predict defect occurrences using the Mel-Frequency Cepstral Coefficients (MFCCs) of the lasermaterial interaction sound. The CNN model is compared to various classic machine learning models trained on the denoised acoustic dataset and raw acoustic dataset. The validation results shows that the CNN model trained on the denoised dataset outperforms others with the highest overall accuracy (89%), keyhole pore prediction accuracy (93%), and AUC-ROC score (98%). Furthermore, the trained CNN model can be deployed into an in-house developed software platform for online quality monitoring. The proposed strategy is the first study to use acoustic signals with deep learning for insitu defect detection in LDED process.

【3】 ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit标题：ESPnet-ST-v2：多用途口语翻译工具包链接：http://arxiv.org/pdf/2304.04596v1

作者：Brian Yan,Jiatong Shi,Yun Tang,Hirofumi Inaguma,Yifan Peng,Siddharth Dalmia,Peter Polák,Patrick Fernandes,Dan Berrebbi,Tomoki Hayashi,Xiaohui Zhang,Zhaoheng Ni,Moto Hira,Soumi Maiti,Juan Pino,Shinji Watanabe机构：Carnegie Mellon University, Meta AI, Charles University, Nagoya University, Johns Hopkins University摘要：ESPnet-ST-v2是对开源ESPnet-ST工具包的改进，这是口语翻译社区不断扩大的兴趣所必需的。ESPnet-ST-v2支持1）离线语音到文本翻译（ST），2）同步语音到文本翻译（SST）和3）离线语音到语音翻译（S2 ST）-每个任务都支持各种各样的方法，将ESPnet-ST-v2与其他开源口语翻译工具包区分开来。该工具包提供了最先进的架构，如传感器，混合CTC 注意力，多解码器与可搜索的中间，时间同步块CTC 注意力，Translatotron模型和直接离散单元模型。在本文中，我们描述了ESPnet-ST-v2背后的整体设计，每个任务的示例模型和性能基准测试，可在www.example.com上公开获取https: github.com espnet espnet。摘要：ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitated by the broadening interests of the spoken language translation community. ESPnet-ST-v2 supports 1) offline speech-to-text translation (ST), 2) simultaneous speech-to-text translation (SST), and 3) offline speech-to-speech translation (S2ST) -- each task is supported with a wide variety of approaches, differentiating ESPnet-ST-v2 from other open source spoken language translation toolkits. This toolkit offers state-of-the-art architectures such as transducers, hybrid CTC attention, multi-decoders with searchable intermediates, time-synchronous blockwise CTC attention, Translatotron models, and direct discrete unit models. In this paper, we describe the overall design, example models for each task, and performance benchmarking behind ESPnet-ST-v2, which is publicly available at https: github.com espnet espnet.

【4】 Oh, Jeez! or Uh-huh? A Listener-aware Backchannel Predictor on ASR Transcriptions标题：哦，天哪！或者是啊哈？一种听众感知的ASR转录反向通道预测器链接：http://arxiv.org/pdf/2304.04478v1

作者：Daniel Ortega,Chia-Yu Li,Ngoc Thang Vu机构：Institute for Natural Language Processing (IMS), University of Stuttgart, Germany备注：Published in ICASSP 2020摘要：本文介绍了我们的最新研究建模会话中的反向通道。出于积极主动的backchanneling理论，我们的目标是开发一个系统，作为一个积极主动的听众插入backchannels，如continuers和评估，影响扬声器。我们的模型不仅考虑到词汇和声学线索，但也引入了简单而新颖的想法，使用...嵌入来模仿不同的反向行为。我们在Switchboard基准数据集上的实验结果表明，在这项任务中，声学线索比词汇线索更重要，并且它们与收听者嵌入的组合在手动转录和自动生成的转录上效果最好。摘要：This paper presents our latest investigation on modeling backchannel in conversations. Motivated by a proactive backchanneling theory, we aim at developing a system which acts as a proactive listener by inserting backchannels, such as continuers and assessment, to influence speakers. Our model takes into account not only lexical and acoustic cues, but also introduces the simple and novel idea of using listener embeddings to mimic different backchanneling behaviours. Our experimental results on the Switchboard benchmark dataset reveal that acoustic cues are more important than lexical cues in this task and their combination with listener embeddings works best on both, manual transcriptions and automatically generated transcriptions.

【5】 Unsupervised Speech Representation Pooling Using Vector Quantization标题：使用矢量量化的无监督语音表示池链接：http://arxiv.org/pdf/2304.03940v1

作者：Jeongkyun Park,Kwanghee Choi,Hyunjun Heo,Hyung-Min Park机构：Sogang University摘要：随着来自大规模自监督模型的通用语音表示的出现，将单个模型应用于多个下游任务正在成为事实上的方法。但是，统筹问题依然存在;语音表示的长度是固有可变的。通常使用朴素平均池化，即使它忽略了语音的特征，例如不同长度的音素。因此，我们设计了一种新的池化方法，通过矢量量化，不需要额外的训练，不像基于注意力的池化压缩声学相似的表示。此外，我们评估了各种自监督模型上的各种无监督池化方法。我们收集分散在语音和文本中的各种方法，以评估各种任务：关键字定位、说话者识别、意图分类和情感识别。最后，我们定量和定性分析我们的方法，比较它与监督池方法。摘要：With the advent of general-purpose speech representations from large-scale self-supervised models, applying a single model to multiple downstream tasks is becoming a de-facto approach. However, the pooling problem remains; the length of speech representations is inherently variable. The naive average pooling is often used, even though it ignores the characteristics of speech, such as differently lengthed phonemes. Hence, we design a novel pooling method to squash acoustically similar representations via vector quantization, which does not require additional training, unlike attention-based pooling. Further, we evaluate various unsupervised pooling methods on various self-supervised models. We gather diverse methods scattered around speech and text to evaluate on various tasks: keyword spotting, speaker identification, intent classification, and emotion recognition. Finally, we quantitatively and qualitatively analyze our method, comparing it with supervised pooling methods.

【6】 An Empirical Study and Improvement for Speech Emotion Recognition标题：语音情感识别的实验研究与改进链接：http://arxiv.org/pdf/2304.03899v1

作者：Zhen Wu,Yizhe Lu,Xinyu Dai机构：National Key Laboratory for Novel Software Technology, Nanjing University, Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing备注：Accepted by ICASSP 2023摘要：多模态语音情感识别的目的是从音频和文本中检测说话人的情感。以往的研究主要集中在利用先进的网络来建模和融合不同的模态信息，以促进性能，而忽略了不同的融合策略对情感识别的影响。在这个工作中，我们考虑一个简单但重要的问题：如何融合音频和文本模态信息对于该多模态任务更有帮助。此外，我们提出了一个多模态情感识别模型的角度损失。实证结果表明，我们的方法在IEMOCAP数据集上获得了新的最先进的结果。深入的分析解释了为什么改进后的模型可以实现改进并优于基线。摘要：Multimodal speech emotion recognition aims to detect speakers' emotions from audio and text. Prior works mainly focus on exploiting advanced networks to model and fuse different modality information to facilitate performance, while neglecting the effect of different fusion strategies on emotion recognition. In this work, we consider a simple yet important problem: how to fuse audio and text modality information is more helpful for this multimodal task. Further, we propose a multimodal emotion recognition model improved by perspective loss. Empirical results show our method obtained new state-of-the-art results on the IEMOCAP dataset. The in-depth analysis explains why the improved model can achieve improvements and outperforms baselines.

【7】 About Voice: A Longitudinal Study of Speaker Recognition Dataset Dynamics标题：关于语音：说话人识别数据集动态的纵向研究链接：http://arxiv.org/pdf/2304.03858v1

作者：Casandra Rusti,Anna Leschanowsky,Carolyn Quinlan,Michaela Pnacek,Lauriane Gorce,Wiebke,Hutiri机构：University of Southern California, University of Toronto, York University备注：14 pages (23 with References and Appendix)摘要：与人脸识别一样，说话人识别被广泛用于各种行业的基于语音的生物识别，包括银行、教育、招聘、移民、执法、医疗保健和福利。然而，虽然数据集评估和审计改善了计算机视觉和人脸识别中的数据实践，但说话人识别中的数据实践基本上没有受到质疑。我们的研究旨在通过探索数据集的使用如何随着时间的推移而演变，以及这对说话人识别系统中的偏见和公平性有什么影响，来解决这一差距。以往的研究已经证明了存在的历史，代表性和测量偏差，在流行的说话人识别基准。在本文中，我们对2012年至2021年用于训练和评估的说话人识别数据集进行了纵向研究。我们调查了近700篇论文，以调查社区对数据集的采用以及在说话人识别方法过渡到广泛采用深度神经网络的关键时期内使用的变化。我们的研究确定了该领域最常用的数据集，检查了它们的使用模式，并评估了它们影响偏见，公平性和其他道德问题的属性。我们的研究结果表明，进一步研究的道德和公平的说话人识别技术的领域。摘要：Like face recognition, speaker recognition is widely used for voice-based biometric identification in a broad range of industries, including banking, education, recruitment, immigration, law enforcement, healthcare, and well-being. However, while dataset evaluations and audits have improved data practices in computer vision and face recognition, the data practices in speaker recognition have gone largely unquestioned. Our research aims to address this gap by exploring how dataset usage has evolved over time and what implications this has on bias and fairness in speaker recognition systems. Previous studies have demonstrated the presence of historical, representation, and measurement biases in popular speaker recognition benchmarks. In this paper, we present a longitudinal study of speaker recognition datasets used for training and evaluation from 2012 to 2021. We survey close to 700 papers to investigate community adoption of datasets and changes in usage over a crucial time period where speaker recognition approaches transitioned to the widespread adoption of deep neural networks. Our study identifies the most commonly used datasets in the field, examines their usage patterns, and assesses their attributes that affect bias, fairness, and other ethical concerns. Our findings suggest areas for further research on the ethics and fairness of speaker recognition technology.

【8】 Leveraging Neural Representations for Audio Manipulation标题：利用神经表示进行音频处理链接：http://arxiv.org/pdf/2304.04394v1

作者：Scott H. Hawley,Christian J. Steinmetz机构：Belmont University, Nashville, TN, USA, Centre for Digital Music, Queen Mary University of London, UK, Harmonai, USA备注：Accepted as Express Paper for AES Europe 2023, this https URL摘要：我们研究应用音频操作使用预训练的基于神经网络的自动编码器作为替代传统的信号处理方法，因为前者可以提供更大的语义或感知组织。为了建立这种方法的潜力，我们首先建立，如果从这些模型的表示编码信息的操纵。我们进行了实验，并使用来自两个不同的预训练自动编码器的表示生成可视化。我们的研究结果表明，虽然一些关于音频操作的信息被编码，但这些信息是有限的，并且以非平凡的方式被编码。这是支持我们的尝试，可视化这些表示，这表明，轨迹表示常见的操作通常是非线性和内容相关的，即使是线性信号操作。因此，目前尚不清楚这些预训练的自动编码器如何用于操纵音频信号，然而，我们的结果表明，这可能是由于缺乏与常见音频操纵的解纠缠。摘要：We investigate applying audio manipulations using pretrained neural network-based autoencoders as an alternative to traditional signal processing methods, since the former may provide greater semantic or perceptual organization. To establish the potential of this approach, we first establish if representations from these models encode information about manipulations. We carry out experiments and produce visualizations using representations from two different pretrained autoencoders. Our findings indicate that, while some information about audio manipulations is encoded, this information is both limited and encoded in a non-trivial way. This is supported by our attempts to visualize these representations, which demonstrated that trajectories of representations for common manipulations are typically nonlinear and content dependent, even for linear signal manipulations. As a result, it is not yet clear how these pretrained autoencoders can be used to manipulate audio signals, however, our results indicate this may be due to the lack of disentanglement with respect to common audio manipulations.

【9】 An investigation of speaker independent phrase break models in End-to-End TTS systems标题：端到端TTS系统中非特定人短语中断模型的研究链接：http://arxiv.org/pdf/2304.04157v1

作者：Anandaswarup Vadapalli机构：Speech Processing Lab, International Institute of Information Technology Hyderabad, Hyderabad , India备注：Submitted for review to IEEE Access摘要：本文介绍了我们在端到端TTS系统背景下的短语中断预测方面的工作，其动机是以下问题：（i）在端到端TTS系统中合并显式短语模型是否有任何效用？以及（ii）您如何评估端到端TTS系统中的短语模型的有效性？特别是，短语中断预测模型的实用性和有效性进行了评估，在儿童故事合成的背景下，使用听众的理解。我们通过感知听力评估表明，有一个明显的偏好，预测后合成的故事短语休息的位置，使用一个训练有素的措辞模型，在故事直接合成，而不预测短语休息的位置。摘要：This paper presents our work on phrase break prediction in the context of end-to-end TTS systems, motivated by the following questions: (i) Is there any utility in incorporating an explicit phrasing model in an end-to-end TTS system?, and (ii) How do you evaluate the effectiveness of a phrasing model in an end-to-end TTS system? In particular, the utility and effectiveness of phrase break prediction models are evaluated in in the context of childrens story synthesis, using listener comprehension. We show by means of perceptual listening evaluations that there is a clear preference for stories synthesized after predicting the location of phrase breaks using a trained phrasing model, over stories directly synthesized without predicting the location of phrase breaks.
eess.AS音频处理

【1】 Leveraging Neural Representations for Audio Manipulation标题：利用神经表示进行音频处理链接：http://arxiv.org/pdf/2304.04394v1

*与cs.SD语音【8】为同一篇

作者：Scott H. Hawley,Christian J. Steinmetz机构：Belmont University, Nashville, TN, USA, Centre for Digital Music, Queen Mary University of London, UK, Harmonai, USA备注：Accepted as Express Paper for AES Europe 2023, this https URL摘要：我们研究应用音频操作使用预训练的基于神经网络的自动编码器作为替代传统的信号处理方法，因为前者可以提供更大的语义或感知组织。为了建立这种方法的潜力，我们首先建立，如果从这些模型的表示编码信息的操纵。我们进行了实验，并使用来自两个不同的预训练自动编码器的表示生成可视化。我们的研究结果表明，虽然一些关于音频操作的信息被编码，但这些信息是有限的，并且以非平凡的方式被编码。这是支持我们的尝试，可视化这些表示，这表明，轨迹表示常见的操作通常是非线性和内容相关的，即使是线性信号操作。因此，目前尚不清楚这些预训练的自动编码器如何用于操纵音频信号，然而，我们的结果表明，这可能是由于缺乏与常见音频操纵的解纠缠。摘要：We investigate applying audio manipulations using pretrained neural network-based autoencoders as an alternative to traditional signal processing methods, since the former may provide greater semantic or perceptual organization. To establish the potential of this approach, we first establish if representations from these models encode information about manipulations. We carry out experiments and produce visualizations using representations from two different pretrained autoencoders. Our findings indicate that, while some information about audio manipulations is encoded, this information is both limited and encoded in a non-trivial way. This is supported by our attempts to visualize these representations, which demonstrated that trajectories of representations for common manipulations are typically nonlinear and content dependent, even for linear signal manipulations. As a result, it is not yet clear how these pretrained autoencoders can be used to manipulate audio signals, however, our results indicate this may be due to the lack of disentanglement with respect to common audio manipulations.

【2】 An investigation of speaker independent phrase break models in End-to-End TTS systems标题：端到端TTS系统中非特定人短语中断模型的研究链接：http://arxiv.org/pdf/2304.04157v1

*与cs.SD语音【9】为同一篇

作者：Anandaswarup Vadapalli机构：Speech Processing Lab, International Institute of Information Technology Hyderabad, Hyderabad , India备注：Submitted for review to IEEE Access摘要：本文介绍了我们在端到端TTS系统背景下的短语中断预测方面的工作，其动机是以下问题：（i）在端到端TTS系统中合并显式短语模型是否有任何效用？以及（ii）您如何评估端到端TTS系统中的短语模型的有效性？特别是，短语中断预测模型的实用性和有效性进行了评估，在儿童故事合成的背景下，使用听众的理解。我们通过感知听力评估表明，有一个明显的偏好，预测后合成的故事短语休息的位置，使用一个训练有素的措辞模型，在故事直接合成，而不预测短语休息的位置。摘要：This paper presents our work on phrase break prediction in the context of end-to-end TTS systems, motivated by the following questions: (i) Is there any utility in incorporating an explicit phrasing model in an end-to-end TTS system?, and (ii) How do you evaluate the effectiveness of a phrasing model in an end-to-end TTS system? In particular, the utility and effectiveness of phrase break prediction models are evaluated in in the context of childrens story synthesis, using listener comprehension. We show by means of perceptual listening evaluations that there is a clear preference for stories synthesized after predicting the location of phrase breaks using a trained phrasing model, over stories directly synthesized without predicting the location of phrase breaks.

【3】 Enhancing Speech-to-Speech Translation with Multiple TTS Targets标题：利用多个TTS目标增强语音到语音的翻译链接：http://arxiv.org/pdf/2304.04618v1

*与cs.SD语音【1】为同一篇

作者：Jiatong Shi,Yun Tang,Ann Lee,Hirofumi Inaguma,Changhan Wang,Juan Pino,Shinji Watanabe机构：Carnegie Mellon University, Meta AI摘要：人们已经知道，直接语音到语音翻译（S2 ST）模型通常遭受的数据稀缺的问题，因为有限的现有的平行材料的源和目标语音。因此，为了训练直接S2 ST系统，以前的工作通常利用文本到语音（TTS）系统通过增强来自语音到文本翻译（S2 TT）的数据来生成目标语言的样本。然而，有一个有限的调查如何合成的目标语音会影响S2 ST模型。在这项工作中，我们分析了直接S2 ST模型的合成目标语音的变化的效果。我们发现，简单地结合不同的TTS系统的目标语音可以潜在地提高S2 ST性能。接下来，我们还提出了一个多任务框架，共同优化S2 ST系统与来自不同的TTS系统的多个目标。大量的实验表明，我们提出的框架实现了一致的改进（2.8 BLEU）在Fisher西班牙语-英语数据集的基线。摘要：It has been known that direct speech-to-speech translation (S2ST) models usually suffer from the data scarcity issue because of the limited existing parallel materials for both source and target speech. Therefore to train a direct S2ST system, previous works usually utilize text-to-speech (TTS) systems to generate samples in the target language by augmenting the data from speech-to-text translation (S2TT). However, there is a limited investigation into how the synthesized target speech would affect the S2ST models. In this work, we analyze the effect of changing synthesized target speech for direct S2ST models. We find that simply combining the target speech from different TTS systems can potentially improve the S2ST performances. Following that, we also propose a multi-task framework that jointly optimizes the S2ST system with multiple targets from different TTS systems. Extensive experiments demonstrate that our proposed framework achieves consistent improvements (2.8 BLEU) over the baselines on the Fisher Spanish-English dataset.

【4】 In-situ crack and keyhole pore detection in laser directed energy deposition through acoustic signal and deep learning标题：基于声信号和深度学习的激光定向能沉积裂纹和小孔原位检测链接：http://arxiv.org/pdf/2304.04598v1

*与cs.SD语音【2】为同一篇

作者：Lequn Chen,Xiling Yao,Chaolin Tan,Weiyang He,Jinlong Su,Fei Weng,Youxiang Chew,Nicholas Poh Huat Ng,Seung Ki Moon机构：a Advanced Remanufacturing and Technology Centre (ARTC), ASTAR, Cleantech Loop, Singapore, b School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore备注：36 Pages, 16 Figures, accepted at journal Additive Manufacturing摘要：裂纹和小孔是激光定向能量沉积（LDED）制备的合金中的有害缺陷。激光-材料相互作用声音可以保持关于潜在的复杂物理事件（诸如裂纹扩展和孔隙形成）的信息。然而，由于嘈杂的环境和复杂的信号内容，在LDED基于声学的监测很少受到关注。本文提出了一种新的基于声学的LDED原位缺陷检测策略。本研究的主要贡献是开发一种原位声学信号去噪、特征提取和声音分类管道，该管道采用卷积神经网络（CNN）进行在线缺陷预测。显微镜图像用于识别部件内的裂纹和锁眼孔的位置。缺陷位置与声学信号在空间上配准。各种声学特征对应的无缺陷的区域，裂纹，和小孔孔隙提取和分析，在时域，频域，和时间-频率表示。CNN模型进行训练，以预测缺陷的发生，使用梅尔频率倒谱系数（MFCC）的lasermaterial相互作用的声音。将CNN模型与在去噪声学数据集和原始声学数据集上训练的各种经典机器学习模型进行比较。验证结果表明，在去噪数据集上训练的CNN模型优于其他模型，具有最高的总体准确度（89%），小孔预测准确度（93%）和AUC-ROC评分（98%）。此外，经过训练的CNN模型可以部署到内部开发的软件平台中，用于在线质量监控。所提出的策略是第一次使用具有深度学习的声学信号在LDED过程中进行原位缺陷检测的研究。摘要：Cracks and keyhole pores are detrimental defects in alloys produced by laser directed energy deposition (LDED). Laser-material interaction sound may hold information about underlying complex physical events such as crack propagation and pores formation. However, due to the noisy environment and intricate signal content, acoustic-based monitoring in LDED has received little attention. This paper proposes a novel acoustic-based in-situ defect detection strategy in LDED. The key contribution of this study is to develop an in-situ acoustic signal denoising, feature extraction, and sound classification pipeline that incorporates convolutional neural networks (CNN) for online defect prediction. Microscope images are used to identify locations of the cracks and keyhole pores within a part. The defect locations are spatiotemporally registered with acoustic signal. Various acoustic features corresponding to defect-free regions, cracks, and keyhole pores are extracted and analysed in time-domain, frequency-domain, and time-frequency representations. The CNN model is trained to predict defect occurrences using the Mel-Frequency Cepstral Coefficients (MFCCs) of the lasermaterial interaction sound. The CNN model is compared to various classic machine learning models trained on the denoised acoustic dataset and raw acoustic dataset. The validation results shows that the CNN model trained on the denoised dataset outperforms others with the highest overall accuracy (89%), keyhole pore prediction accuracy (93%), and AUC-ROC score (98%). Furthermore, the trained CNN model can be deployed into an in-house developed software platform for online quality monitoring. The proposed strategy is the first study to use acoustic signals with deep learning for insitu defect detection in LDED process.

【5】 ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit标题：ESPnet-ST-v2：多用途口语翻译工具包链接：http://arxiv.org/pdf/2304.04596v1

*与cs.SD语音【3】为同一篇

作者：Brian Yan,Jiatong Shi,Yun Tang,Hirofumi Inaguma,Yifan Peng,Siddharth Dalmia,Peter Polák,Patrick Fernandes,Dan Berrebbi,Tomoki Hayashi,Xiaohui Zhang,Zhaoheng Ni,Moto Hira,Soumi Maiti,Juan Pino,Shinji Watanabe机构：Carnegie Mellon University, Meta AI, Charles University, Nagoya University, Johns Hopkins University摘要：ESPnet-ST-v2是对开源ESPnet-ST工具包的改进，这是口语翻译社区不断扩大的兴趣所必需的。ESPnet-ST-v2支持1）离线语音到文本翻译（ST），2）同步语音到文本翻译（SST）和3）离线语音到语音翻译（S2 ST）-每个任务都支持各种各样的方法，将ESPnet-ST-v2与其他开源口语翻译工具包区分开来。该工具包提供了最先进的架构，如传感器，混合CTC 注意力，多解码器与可搜索的中间，时间同步块CTC 注意力，Translatotron模型和直接离散单元模型。在本文中，我们描述了ESPnet-ST-v2背后的整体设计，每个任务的示例模型和性能基准测试，可在www.example.com上公开获取https: github.com espnet espnet。摘要：ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitated by the broadening interests of the spoken language translation community. ESPnet-ST-v2 supports 1) offline speech-to-text translation (ST), 2) simultaneous speech-to-text translation (SST), and 3) offline speech-to-speech translation (S2ST) -- each task is supported with a wide variety of approaches, differentiating ESPnet-ST-v2 from other open source spoken language translation toolkits. This toolkit offers state-of-the-art architectures such as transducers, hybrid CTC attention, multi-decoders with searchable intermediates, time-synchronous blockwise CTC attention, Translatotron models, and direct discrete unit models. In this paper, we describe the overall design, example models for each task, and performance benchmarking behind ESPnet-ST-v2, which is publicly available at https: github.com espnet espnet.

【6】 Oh, Jeez! or Uh-huh? A Listener-aware Backchannel Predictor on ASR Transcriptions标题：哦，天哪！或者是啊哈？一种听众感知的ASR转录反向通道预测器链接：http://arxiv.org/pdf/2304.04478v1

*与cs.SD语音【4】为同一篇

作者：Daniel Ortega,Chia-Yu Li,Ngoc Thang Vu机构：Institute for Natural Language Processing (IMS), University of Stuttgart, Germany备注：Published in ICASSP 2020摘要：本文介绍了我们的最新研究建模会话中的反向通道。出于积极主动的backchanneling理论，我们的目标是开发一个系统，作为一个积极主动的听众插入backchannels，如continuers和评估，影响扬声器。我们的模型不仅考虑到词汇和声学线索，但也引入了简单而新颖的想法，使用...嵌入来模仿不同的反向行为。我们在Switchboard基准数据集上的实验结果表明，在这项任务中，声学线索比词汇线索更重要，并且它们与收听者嵌入的组合在手动转录和自动生成的转录上效果最好。摘要：This paper presents our latest investigation on modeling backchannel in conversations. Motivated by a proactive backchanneling theory, we aim at developing a system which acts as a proactive listener by inserting backchannels, such as continuers and assessment, to influence speakers. Our model takes into account not only lexical and acoustic cues, but also introduces the simple and novel idea of using listener embeddings to mimic different backchanneling behaviours. Our experimental results on the Switchboard benchmark dataset reveal that acoustic cues are more important than lexical cues in this task and their combination with listener embeddings works best on both, manual transcriptions and automatically generated transcriptions.

【7】 Unsupervised Speech Representation Pooling Using Vector Quantization标题：使用矢量量化的无监督语音表示池链接：http://arxiv.org/pdf/2304.03940v1

*与cs.SD语音【5】为同一篇

作者：Jeongkyun Park,Kwanghee Choi,Hyunjun Heo,Hyung-Min Park机构：Sogang University摘要：随着来自大规模自监督模型的通用语音表示的出现，将单个模型应用于多个下游任务正在成为事实上的方法。但是，统筹问题依然存在;语音表示的长度是固有可变的。通常使用朴素平均池化，即使它忽略了语音的特征，例如不同长度的音素。因此，我们设计了一种新的池化方法，通过矢量量化，不需要额外的训练，不像基于注意力的池化压缩声学相似的表示。此外，我们评估了各种自监督模型上的各种无监督池化方法。我们收集分散在语音和文本中的各种方法，以评估各种任务：关键字定位、说话者识别、意图分类和情感识别。最后，我们定量和定性分析我们的方法，比较它与监督池方法。摘要：With the advent of general-purpose speech representations from large-scale self-supervised models, applying a single model to multiple downstream tasks is becoming a de-facto approach. However, the pooling problem remains; the length of speech representations is inherently variable. The naive average pooling is often used, even though it ignores the characteristics of speech, such as differently lengthed phonemes. Hence, we design a novel pooling method to squash acoustically similar representations via vector quantization, which does not require additional training, unlike attention-based pooling. Further, we evaluate various unsupervised pooling methods on various self-supervised models. We gather diverse methods scattered around speech and text to evaluate on various tasks: keyword spotting, speaker identification, intent classification, and emotion recognition. Finally, we quantitatively and qualitatively analyze our method, comparing it with supervised pooling methods.

【8】 An Empirical Study and Improvement for Speech Emotion Recognition标题：语音情感识别的实验研究与改进链接：http://arxiv.org/pdf/2304.03899v1

*与cs.SD语音【6】为同一篇

作者：Zhen Wu,Yizhe Lu,Xinyu Dai机构：National Key Laboratory for Novel Software Technology, Nanjing University, Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing备注：Accepted by ICASSP 2023摘要：多模态语音情感识别的目的是从音频和文本中检测说话人的情感。以往的研究主要集中在利用先进的网络来建模和融合不同的模态信息，以促进性能，而忽略了不同的融合策略对情感识别的影响。在这个工作中，我们考虑一个简单但重要的问题：如何融合音频和文本模态信息对于该多模态任务更有帮助。此外，我们提出了一个多模态情感识别模型的角度损失。实证结果表明，我们的方法在IEMOCAP数据集上获得了新的最先进的结果。深入的分析解释了为什么改进后的模型可以实现改进并优于基线。摘要：Multimodal speech emotion recognition aims to detect speakers' emotions from audio and text. Prior works mainly focus on exploiting advanced networks to model and fuse different modality information to facilitate performance, while neglecting the effect of different fusion strategies on emotion recognition. In this work, we consider a simple yet important problem: how to fuse audio and text modality information is more helpful for this multimodal task. Further, we propose a multimodal emotion recognition model improved by perspective loss. Empirical results show our method obtained new state-of-the-art results on the IEMOCAP dataset. The in-depth analysis explains why the improved model can achieve improvements and outperforms baselines.

【9】 About Voice: A Longitudinal Study of Speaker Recognition Dataset Dynamics标题：关于语音：说话人识别数据集动态的纵向研究链接：http://arxiv.org/pdf/2304.03858v1

*与cs.SD语音【7】为同一篇

作者：Casandra Rusti,Anna Leschanowsky,Carolyn Quinlan,Michaela Pnacek,Lauriane Gorce,Wiebke,Hutiri机构：University of Southern California, University of Toronto, York University备注：14 pages (23 with References and Appendix)摘要：与人脸识别一样，说话人识别被广泛用于各种行业的基于语音的生物识别，包括银行、教育、招聘、移民、执法、医疗保健和福利。然而，虽然数据集评估和审计改善了计算机视觉和人脸识别中的数据实践，但说话人识别中的数据实践基本上没有受到质疑。我们的研究旨在通过探索数据集的使用如何随着时间的推移而演变，以及这对说话人识别系统中的偏见和公平性有什么影响，来解决这一差距。以往的研究已经证明了存在的历史，代表性和测量偏差，在流行的说话人识别基准。在本文中，我们对2012年至2021年用于训练和评估的说话人识别数据集进行了纵向研究。我们调查了近700篇论文，以调查社区对数据集的采用以及在说话人识别方法过渡到广泛采用深度神经网络的关键时期内使用的变化。我们的研究确定了该领域最常用的数据集，检查了它们的使用模式，并评估了它们影响偏见，公平性和其他道德问题的属性。我们的研究结果表明，进一步研究的道德和公平的说话人识别技术的领域。摘要：Like face recognition, speaker recognition is widely used for voice-based biometric identification in a broad range of industries, including banking, education, recruitment, immigration, law enforcement, healthcare, and well-being. However, while dataset evaluations and audits have improved data practices in computer vision and face recognition, the data practices in speaker recognition have gone largely unquestioned. Our research aims to address this gap by exploring how dataset usage has evolved over time and what implications this has on bias and fairness in speaker recognition systems. Previous studies have demonstrated the presence of historical, representation, and measurement biases in popular speaker recognition benchmarks. In this paper, we present a longitudinal study of speaker recognition datasets used for training and evaluation from 2012 to 2021. We survey close to 700 papers to investigate community adoption of datasets and changes in usage over a crucial time period where speaker recognition approaches transitioned to the widespread adoption of deep neural networks. Our study identifies the most commonly used datasets in the field, examines their usage patterns, and assesses their attributes that affect bias, fairness, and other ethical concerns. Our findings suggest areas for further research on the ethics and fairness of speaker recognition technology.

机器翻译，仅供参考