今日论文合集：cs.SD语音3篇，eess.AS音频处理4篇。
cs.SD语音
【1】 EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers
标题：EarSpy：通过智能手机耳机扬声器的微小振动监视呼叫者的语音和身份
链接：https://arxiv.org/abs/2212.12151
作者：Ahmed Tanvir Mahdad,Cong Shi,Zhengkun Ye,Tianming Zhao,Yan Wang,Yingying Chen,Nitesh Saxena机构：Texas A&M University, New Jersey Institute of, Technology, Temple University, University of Dayton, Rutgers University, Texas A&M Unviersity摘要：从用户的智能手机窃听是对用户安全和隐私的众所周知的威胁。已有的研究表明，扬声器混响会将语音注入到运动传感器的读数中，从而导致语音窃听。而对耳用扬声器的更具破坏性的攻击，产生小得多的规模振动，被认为是不可能窃听与零许可运动传感器。在本书中，我们将重新审视这条重要的延伸线。我们探讨了智能手机制造商的最新趋势，包括用额外/强大的扬声器代替小型耳用扬声器，并演示了使用运动传感器捕捉这种微小语音振动的可行性。我们调查了这些新型耳用扬声器对内置运动传感器的影响，并检查了从微小振动中引出私人语音信息的潜力。我们设计的EarSpy系统可以成功地检测出单词区域、时域和频域特征，并为每个单词区域生成一个声谱图。我们使用经典的机器学习算法和卷积神经网络对提取的数据进行训练和测试。我们发现性别检测的准确率高达98.66%，说话人检测的准确率为92.6%，数字检测的准确率为56.42%（比随机选择（10%）显著5倍）。我们的结果揭示了使用运动传感器从耳用扬声器窃听电话交谈的潜在威胁。摘要：Eavesdropping from the user's smartphone is a well-known threat to the user's safety and privacy. Existing studies show that loudspeaker reverberation can inject speech into motion sensor readings, leading to speech eavesdropping. While more devastating attacks on ear speakers, which produce much smaller scale vibrations, were believed impossible to eavesdrop with zero-permission motion sensors. In this work, we revisit this important line of reach. We explore recent trends in smartphone manufacturers that include extra/powerful speakers in place of small ear speakers, and demonstrate the feasibility of using motion sensors to capture such tiny speech vibrations. We investigate the impacts of these new ear speakers on built-in motion sensors and examine the potential to elicit private speech information from the minute vibrations. Our designed system EarSpy can successfully detect word regions, time, and frequency domain features and generate a spectrogram for each word region. We train and test the extracted data using classical machine learning algorithms and convolutional neural networks. We found up to 98.66% accuracy in gender detection, 92.6% detection in speaker detection, and 56.42% detection in digit detection (which is 5X more significant than the random selection (10%)). Our result unveils the potential threat of eavesdropping on phone conversations from ear speakers using motion sensors.
【2】 Pushing the performances of ASR models on English and Spanish accents
标题：用英语和西班牙口音推动ASR模特的表演
链接：https://arxiv.org/abs/2212.12048
作者：Pooja Chitkara,Morgane Riviere,Jade Copet,Frank Zhang,Yatharth Saraf摘要：语音到文本模型倾向于针对单个目标重音来训练和评估。这对于英语来说尤其如此，因为来自美国的母语者成为英语的主要基准。在这项工作中，我们将展示如何两个简单的方法：预训练嵌入和辅助分类损失可以改善ASR系统的性能。我们正在寻找尽可能通用的升级，因此我们将探讨它们对几种模型架构和几种语言的影响。摘要：Speech to text models tend to be trained and evaluated against a single target accent. This is especially true for English for which native speakers from the United States became the main benchmark. In this work, we are going to show how two simple methods: pre-trained embeddings and auxiliary classification losses can improve the performance of ASR systems. We are looking for upgrades as universal as possible and therefore we will explore their impact on several models architectures and several languages.
【3】 Fractal Patterns in Music
标题：音乐中的分形图
链接：https://arxiv.org/abs/2212.12497
作者：John McDonough,Andrzej Herczyński机构：Department of Physics, Boston College, MA摘要：如果我们的审美偏好受到自然界分形几何学的影响，那么尺度规律将出现在包括音乐在内的所有艺术形式中。虽然已经提出了多种统计工具来分析声音中的时间序列，但对于音乐中复杂性的最有意义的测量，或者如何首先辨别作品中的分形模式，还没有形成共识。在这里，我们提供了一种新的方法，基于自相似性的旋律线在不同的时间尺度。与最近文献中提出的统计分析方法不同，该方法不依赖于时间窗内的平均，并且具有明显的局部性。分形维数的相应定义是基于时间尺度层次的，并且取决于音乐主题的音调轮廓。新的概念是测试音乐“再现”的康托集和科赫曲线，然后应用于一些精心挑选的精湛的作品跨越五个世纪的音乐制作。摘要：If our aesthetic preferences are affected by fractal geometry of nature, scaling regularities would be expected to appear in all art forms, including music. While a variety of statistical tools have been proposed to analyze time series in sound, no consensus has as yet emerged regarding the most meaningful measure of complexity in music, or how to discern fractal patterns in compositions in the first place. Here we offer a new approach based on self-similarity of the melodic lines recurring at various temporal scales. In contrast to the statistical analyses advanced in recent literature, the proposed method does not depend on averaging within time-windows and is distinctively local. The corresponding definition of the fractal dimension is based on the temporal scaling hierarchy and depends on the tonal contours of the musical motifs. The new concepts are tested on musical 'renditions' of the Cantor Set and Koch Curve, and then applied to a number of carefully selected masterful compositions spanning five centuries of music making.
【1】Large Raw Emotional Dataset with Aggregation Mechanism
标题：具有聚合机制的大型原始情感数据集
链接：https://arxiv.org/abs/2212.12266
作者：Vladimir Kondratenko,Artem Sokolov,Nikolay Karpov,Oleg Kutuzov,Nikita Savushkin,Fyodor Minkin机构：Sber, Russia, Nvidia, Armenia, HSE University, Laboratory of Algorithms and Technologies for Network Analysis, Russia备注：6 pages, 1 figures, submitted to ICASSP 2023摘要：我们提出了一个新的语音情感识别任务的数据集称为杜沙。该语料库包含约350小时的数据、30多万份俄语录音及其抄本。因此，它是当今SER任务中最大的开放式双模态数据采集系统。它使用众包平台进行注释，包括两个子集：演戏和现实生活。与由音频播客组成的不平衡的现实生活部分相比，行动子集具有更平衡的类分布。因此，第一个适合于模型预训练，而第二个则用于微调、模型认可和验证。本文描述了预处理程序、注释和基线模型实验，以展示可以使用杜沙数据集获得的一些实际度量。摘要：We present a new data set for speech emotion recognition (SER) tasks called Dusha. The corpus contains approximately 350 hours of data, more than 300 000 audio recordings with Russian speech and their transcripts. Therefore it is the biggest open bi-modal data collection for SER task nowadays. It is annotated using a crowd-sourcing platform and includes two subsets: acted and real-life. Acted subset has a more balanced class distribution than the unbalanced real-life part consisting of audio podcasts. So the first one is suitable for model pre-training, and the second is elaborated for fine-tuning purposes, model approbation, and validation. This paper describes pre-processing routine, annotation, and experiment with a baseline model to demonstrate some actual metrics which could be obtained with the Dusha data set.
【2】 Fractal Patterns in Music
标题：音乐中的分形图
链接：https://arxiv.org/abs/2212.12497
*与cs.SD语音【3】为同一篇
作者：John McDonough,Andrzej Herczyński机构：Department of Physics, Boston College, MA摘要：如果我们的审美偏好受到自然界分形几何学的影响，那么尺度规律将出现在包括音乐在内的所有艺术形式中。虽然已经提出了多种统计工具来分析声音中的时间序列，但对于音乐中复杂性的最有意义的测量，或者如何首先辨别作品中的分形模式，还没有形成共识。在这里，我们提供了一种新的方法，基于自相似性的旋律线在不同的时间尺度。与最近文献中提出的统计分析方法不同，该方法不依赖于时间窗内的平均，并且具有明显的局部性。分形维数的相应定义是基于时间尺度层次的，并且取决于音乐主题的音调轮廓。新的概念是测试音乐“再现”的康托集和科赫曲线，然后应用于一些精心挑选的精湛的作品跨越五个世纪的音乐制作。摘要：If our aesthetic preferences are affected by fractal geometry of nature, scaling regularities would be expected to appear in all art forms, including music. While a variety of statistical tools have been proposed to analyze time series in sound, no consensus has as yet emerged regarding the most meaningful measure of complexity in music, or how to discern fractal patterns in compositions in the first place. Here we offer a new approach based on self-similarity of the melodic lines recurring at various temporal scales. In contrast to the statistical analyses advanced in recent literature, the proposed method does not depend on averaging within time-windows and is distinctively local. The corresponding definition of the fractal dimension is based on the temporal scaling hierarchy and depends on the tonal contours of the musical motifs. The new concepts are tested on musical 'renditions' of the Cantor Set and Koch Curve, and then applied to a number of carefully selected masterful compositions spanning five centuries of music making.
【3】 EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers
标题：EarSpy：通过智能手机耳机扬声器的微小振动监视呼叫者的语音和身份
链接：https://arxiv.org/abs/2212.12151
*与cs.SD语音【1】为同一篇
作者：Ahmed Tanvir Mahdad,Cong Shi,Zhengkun Ye,Tianming Zhao,Yan Wang,Yingying Chen,Nitesh Saxena机构：Texas A&M University, New Jersey Institute of, Technology, Temple University, University of Dayton, Rutgers University, Texas A&M Unviersity摘要：从用户的智能手机窃听是对用户安全和隐私的众所周知的威胁。已有的研究表明，扬声器混响会将语音注入到运动传感器的读数中，从而导致语音窃听。而对耳用扬声器的更具破坏性的攻击，产生小得多的规模振动，被认为是不可能窃听与零许可运动传感器。在本书中，我们将重新审视这条重要的延伸线。我们探讨了智能手机制造商的最新趋势，包括用额外/强大的扬声器代替小型耳用扬声器，并演示了使用运动传感器捕捉这种微小语音振动的可行性。我们调查了这些新型耳用扬声器对内置运动传感器的影响，并检查了从微小振动中引出私人语音信息的潜力。我们设计的EarSpy系统可以成功地检测出单词区域、时域和频域特征，并为每个单词区域生成一个声谱图。我们使用经典的机器学习算法和卷积神经网络对提取的数据进行训练和测试。我们发现性别检测的准确率高达98.66%，说话人检测的准确率为92.6%，数字检测的准确率为56.42%（比随机选择（10%）显著5倍）。我们的结果揭示了使用运动传感器从耳用扬声器窃听电话交谈的潜在威胁。摘要：Eavesdropping from the user's smartphone is a well-known threat to the user's safety and privacy. Existing studies show that loudspeaker reverberation can inject speech into motion sensor readings, leading to speech eavesdropping. While more devastating attacks on ear speakers, which produce much smaller scale vibrations, were believed impossible to eavesdrop with zero-permission motion sensors. In this work, we revisit this important line of reach. We explore recent trends in smartphone manufacturers that include extra/powerful speakers in place of small ear speakers, and demonstrate the feasibility of using motion sensors to capture such tiny speech vibrations. We investigate the impacts of these new ear speakers on built-in motion sensors and examine the potential to elicit private speech information from the minute vibrations. Our designed system EarSpy can successfully detect word regions, time, and frequency domain features and generate a spectrogram for each word region. We train and test the extracted data using classical machine learning algorithms and convolutional neural networks. We found up to 98.66% accuracy in gender detection, 92.6% detection in speaker detection, and 56.42% detection in digit detection (which is 5X more significant than the random selection (10%)). Our result unveils the potential threat of eavesdropping on phone conversations from ear speakers using motion sensors.
【4】 Pushing the performances of ASR models on English and Spanish accents
标题：用英语和西班牙口音推动ASR模特的表演
链接：https://arxiv.org/abs/2212.12048
*与cs.SD语音【2】为同一篇
作者：Pooja Chitkara,Morgane Riviere,Jade Copet,Frank Zhang,Yatharth Saraf摘要：语音到文本模型倾向于针对单个目标重音来训练和评估。这对于英语来说尤其如此，因为来自美国的母语者成为英语的主要基准。在这项工作中，我们将展示如何两个简单的方法：预训练嵌入和辅助分类损失可以改善ASR系统的性能。我们正在寻找尽可能通用的升级，因此我们将探讨它们对几种模型架构和几种语言的影响。摘要：Speech to text models tend to be trained and evaluated against a single target accent. This is especially true for English for which native speakers from the United States became the main benchmark. In this work, we are going to show how two simple methods: pre-trained embeddings and auxiliary classification losses can improve the performance of ASR systems. We are looking for upgrades as universal as possible and therefore we will explore their impact on several models architectures and several languages.
机器翻译，仅供参考
每日语音速递