今日论文合集：cs.SD语音2篇，eess.AS音频处理4篇。
cs.SD语音

【1】 A Change of Heart: Improving Speech Emotion Recognition through  Speech-to-Text Modality Conversion标题：改变主意：通过语音到文本的通道转换来提高语音情感识别链接：https://arxiv.org/abs/2307.11584

作者：Zeinab Sadat Taghavi,Ali Satvaty,Hossein Sameti机构：Department of Computer Engineering, Sharif University of Technology, Tehran, Iran摘要：语音情感识别（SER）是一项具有挑战性的任务。在本文中，我们引入了一个模态转换的概念，旨在提高MELD数据集上的情感识别性能。我们通过两个实验来评估我们的方法：第一种是一种名为模态转换的方法，它采用自动语音识别（ASR）系统，然后是文本分类器;第二，我们假设完美的ASR输出，并研究模态转换对SER的影响，这种方法被称为Modality-Conversion++。我们的研究结果表明，第一种方法产生了实质性的结果，而第二种方法优于国家的最先进的（SOTA）语音为基础的方法在SER加权F1（WF 1）评分的MELD数据集。这项研究突出了潜在的模态转换的任务，可以在其他方式进行。摘要：Speech Emotion Recognition (SER) is a challenging task. In this paper, we introduce a modality conversion concept aimed at enhancing emotion recognition performance on the MELD dataset. We assess our approach through two experiments: first, a method named Modality-Conversion that employs automatic speech recognition (ASR) systems, followed by a text classifier; second, we assume perfect ASR output and investigate the impact of modality conversion on SER, this method is called Modality-Conversion++. Our findings indicate that the first method yields substantial results, while the second method outperforms state-of-the-art (SOTA) speech-based approaches in terms of SER weighted-F1 (WF1) score on the MELD dataset. This research highlights the potential of modality conversion for tasks that can be conducted in alternative modalities.

【2】 Multi-modal Hate Speech Detection using Machine Learning标题：基于机器学习的多模式仇恨语音检测链接：https://arxiv.org/abs/2307.11519
作者：Fariha Tahosin Boishakhi,Ponkoj Chandra Shill,Md. Golam Rabiul Alam机构：Computer Science and Engineering, BRAC University, Dhaka , Bangladesh, Dhaka, Bangladesh备注：5 pages, 2 figures, conference摘要：随着互联网用户和媒体内容的不断增长，很难在音频和视频中追踪仇恨言论。将视频或音频转换为文本并不能准确地检测仇恨言论，因为人类有时会使用仇恨词语作为幽默或愉快的感觉，并且还使用不同的语音语调或在视频中显示不同的动作。最先进的仇恨言论检测模型大多是在单一模态上开发的。在这项研究中，多模态系统的组合方法已被提出来检测仇恨言论从视频内容中提取的特征图像，从音频，文本中提取的特征值，并使用机器学习和自然语言处理。摘要：With the continuous growth of internet users and media content, it is very hard to track down hateful speech in audio and video. Converting video or audio into text does not detect hate speech accurately as human sometimes uses hateful words as humorous or pleasant in sense and also uses different voice tones or show different action in the video. The state-ofthe-art hate speech detection models were mostly developed on a single modality. In this research, a combined approach of multimodal system has been proposed to detect hate speech from video contents by extracting feature images, feature values extracted from the audio, text and used machine learning and Natural language processing.
eess.AS音频处理
【1】 Topic Identification For Spontaneous Speech: Enriching Audio Features  With Embedded Linguistic Information标题：自发性语音话题识别：嵌入语言信息丰富音频特征链接：https://arxiv.org/abs/2307.11450
作者：Dejan Porjazovski,Tamás Grósz,Mikko Kurimo机构：Aalto University, Espoo, Finland, Tam´as Gr´osz备注：Accepted to EUSIPCO 2023摘要：来自音频的传统主题识别解决方案依赖于自动语音识别系统（ASR）来产生用作对基于文本的模型的输入的转录。这些方法在高资源场景中工作良好，其中有足够的数据来训练管道的两个组件。然而，在低资源的情况下，ASR系统，即使可用，产生低质量的成绩单，导致一个坏的基于文本的分类器。此外，包含犹豫的自发语音可以进一步降低ASR模型的性能。在本文中，我们调查的替代方案，标准的纯文本的解决方案，通过比较音频和混合技术，共同利用文本和音频功能。自发的芬兰语语音模型评估表明，纯粹基于音频的解决方案是一个可行的选择时，ASR组件不可用，而混合多模态解决方案实现了最佳效果。摘要：Traditional topic identification solutions from audio rely on an automatic speech recognition system (ASR) to produce transcripts used as input to a text-based model. These approaches work well in high-resource scenarios, where there are sufficient data to train both components of the pipeline. However, in low-resource situations, the ASR system, even if available, produces low-quality transcripts, leading to a bad text-based classifier. Moreover, spontaneous speech containing hesitations can further degrade the performance of the ASR model. In this paper, we investigate alternatives to the standard text-only solutions by comparing audio-only and hybrid techniques of jointly utilising text and audio features. The models evaluated on spontaneous Finnish speech demonstrate that purely audio-based solutions are a viable option when ASR components are not available, while the hybrid multi-modal solutions achieve the best results.
【2】 A Change of Heart: Improving Speech Emotion Recognition through  Speech-to-Text Modality Conversion标题：改变主意：通过语音到文本的通道转换来提高语音情感识别链接：https://arxiv.org/abs/2307.11584
作者：Zeinab Sadat Taghavi,Ali Satvaty,Hossein Sameti机构：Department of Computer Engineering, Sharif University of Technology, Tehran, Iran摘要：语音情感识别（SER）是一项具有挑战性的任务。在本文中，我们引入了一个模态转换的概念，旨在提高MELD数据集上的情感识别性能。我们通过两个实验来评估我们的方法：第一种是一种名为模态转换的方法，它采用自动语音识别（ASR）系统，然后是文本分类器;第二，我们假设完美的ASR输出，并研究模态转换对SER的影响，这种方法被称为Modality-Conversion++。我们的研究结果表明，第一种方法产生了实质性的结果，而第二种方法优于国家的最先进的（SOTA）语音为基础的方法在SER加权F1（WF 1）评分的MELD数据集。这项研究突出了潜在的模态转换的任务，可以在其他方式进行。摘要：Speech Emotion Recognition (SER) is a challenging task. In this paper, we introduce a modality conversion concept aimed at enhancing emotion recognition performance on the MELD dataset. We assess our approach through two experiments: first, a method named Modality-Conversion that employs automatic speech recognition (ASR) systems, followed by a text classifier; second, we assume perfect ASR output and investigate the impact of modality conversion on SER, this method is called Modality-Conversion++. Our findings indicate that the first method yields substantial results, while the second method outperforms state-of-the-art (SOTA) speech-based approaches in terms of SER weighted-F1 (WF1) score on the MELD dataset. This research highlights the potential of modality conversion for tasks that can be conducted in alternative modalities.
【3】 Multi-modal Hate Speech Detection using Machine Learning标题：基于机器学习的多模式仇恨语音检测链接：https://arxiv.org/abs/2307.11519
作者：Fariha Tahosin Boishakhi,Ponkoj Chandra Shill,Md. Golam Rabiul Alam机构：Computer Science and Engineering, BRAC University, Dhaka , Bangladesh, Dhaka, Bangladesh备注：5 pages, 2 figures, conference摘要：随着互联网用户和媒体内容的不断增长，很难在音频和视频中追踪仇恨言论。将视频或音频转换为文本并不能准确地检测仇恨言论，因为人类有时会使用仇恨词语作为幽默或愉快的感觉，并且还使用不同的语音语调或在视频中显示不同的动作。最先进的仇恨言论检测模型大多是在单一模态上开发的。在这项研究中，多模态系统的组合方法已被提出来检测仇恨言论从视频内容中提取的特征图像，从音频，文本中提取的特征值，并使用机器学习和自然语言处理。摘要：With the continuous growth of internet users and media content, it is very hard to track down hateful speech in audio and video. Converting video or audio into text does not detect hate speech accurately as human sometimes uses hateful words as humorous or pleasant in sense and also uses different voice tones or show different action in the video. The state-ofthe-art hate speech detection models were mostly developed on a single modality. In this research, a combined approach of multimodal system has been proposed to detect hate speech from video contents by extracting feature images, feature values extracted from the audio, text and used machine learning and Natural language processing.
【4】 MeetEval: A Toolkit for Computation of Word Error Rates for Meeting  Transcription Systems标题：MeetEval：用于计算满足转录系统的单词错误率的工具包链接：https://arxiv.org/abs/2307.11394
作者：Thilo von Neumann,Christoph Boeddeker,Marc Delcroix,Reinhold Haeb-Umbach机构：Paderborn University, Germany, NTT Corporation, Japan备注：Accepted for presentation at the Chime7 workshop 2023摘要：MeetEval是一个开源工具包，用于评估各种会议转录系统。它提供了一个统一的接口，用于计算常用的误字率（WER），特别是cpWER，ORC WER和MIMO WER以及其他WER定义。我们扩展了cpWER计算的时间约束，以确保只有词被确定为正确时，时间对齐是合理的。这导致假设串与参考串的匹配的更好的质量，其更接近地类似于实际转录质量，并且如果系统提供差的时间注释，则系统被惩罚。由于字级定时信息通常不可用，因此我们提出了一种从段级定时近似精确字级定时的方法（例如，句子），并且示出近似导致类似的WER作为与精确单词级注释的匹配。同时，时间约束导致匹配算法的加速，这超过了由处理时间戳引起的额外开销。摘要：MeetEval is an open-source toolkit to evaluate all kinds of meeting transcription systems. It provides a unified interface for the computation of commonly used Word Error Rates (WERs), specifically cpWER, ORC WER and MIMO WER along other WER definitions. We extend the cpWER computation by a temporal constraint to ensure that only words are identified as correct when the temporal alignment is plausible. This leads to a better quality of the matching of the hypothesis string to the reference string that more closely resembles the actual transcription quality, and a system is penalized if it provides poor time annotations. Since word-level timing information is often not available, we present a way to approximate exact word-level timings from segment-level timings (e.g., a sentence) and show that the approximation leads to a similar WER as a matching with exact word-level annotations. At the same time, the time constraint leads to a speedup of the matching algorithm, which outweighs the additional overhead caused by processing the time stamps.
机器翻译，仅供参考