【1】 Fall Detection from Audios with Audio Transformers
标题：使用Audio Transformers从音频中检测跌倒
链接：https://arxiv.org/abs/2208.10659
作者：Prabhjot Kaur,Qifan Wang,Weisong Shi机构：Wayne State University, Detroit, Michigan, USA, Meta AI, Menlo Park, California, USA摘要：针对老年人的跌倒检测是一个经过充分研究的问题，提出了几种解决方案，包括可穿戴和非可穿戴技术。虽然现有技术具有极好的检测率，但是由于需要佩戴设备和用户隐私问题，它们缺乏目标人群的采用。本文提出了一种新颖的、非穿戴式、非侵入式、可扩展的跌倒检测解决方案，部署在一个带有麦克风的自主移动机器人上。所提出的方法使用在人们家中记录的环境声音输入。我们特别针对浴室环境，因为它非常容易跌倒，并且现有技术无法在不危及用户隐私的情况下部署。本工作开发了Transformer基于Transformer架构的解决方案，该架构从浴室获取噪声声音输入，并将其分类为落音/非落音类，准确度为0.8673。此外，所提出的方法可扩展到除浴室之外的其它室内环境，并且适合于部署在养老院、医院和康复机构中，而不需要用户佩戴任何设备或由传感器持续“监视”。摘要：Fall detection for the elderly is a well-researched problem with several proposed solutions, including wearable and non-wearable techniques. While the existing techniques have excellent detection rates, their adoption by the target population is lacking due to the need for wearing devices and user privacy concerns. Our paper provides a novel, non-wearable, non-intrusive, and scalable solution for fall detection, deployed on an autonomous mobile robot equipped with a microphone. The proposed method uses ambient sound input recorded in people's homes. We specifically target the bathroom environment as it is highly prone to falls and where existing techniques cannot be deployed without jeopardizing user privacy. The present work develops a solution based on a Transformer architecture that takes noisy sound input from bathrooms and classifies it into fall/no-fall class with an accuracy of 0.8673. Further, the proposed approach is extendable to other indoor environments, besides bathrooms and is suitable for deploying in elderly homes, hospitals, and rehabilitation facilities without requiring the user to wear any device or be constantly "watched" by the sensors.
【2】 Concurrent Validity of Automatic Speech and Pause Measures During Passage Reading in ALS
标题：肌萎缩侧索硬化症患者短文阅读中自动语音和停顿测量的同时效度
链接：https://arxiv.org/abs/2208.10597
作者：Saeid Alavi Naeini,Leif Simmatis,Yana Yunusova,Babak Taati机构：Kite Research Institute, Toronto Rehabilitation Institute – University Health Network, Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada, Department of Speech-Language Pathology, University of Toronto, Toronto, Canada备注：2022 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI)摘要：分析肌萎缩侧索硬化症（ALS）患者的语音测量值可以为早期诊断和跟踪疾病进展提供必要的信息。然而，目前用于提取语音和停顿特征的方法是手动或半自动的，这使得它们耗时且劳动强度大。语音—文本对齐算法的出现为ALS患者的语音测量提供了廉价、自动和准确的分析机会。需要对照当前的黄金标准方法来验证由这些算法计算的语音和停顿特征。在本研究中，我们从646个ALS患者和健康对照者进行段落阅读的音频文件中提取了8个语音/停顿特征。两个预先训练的强制对齐模型—一个使用Transformers，另一个使用高斯混合/隐马尔可夫结构—用于自动特征提取。然后用半自动语音/停顿分析软件对结果进行验证，并根据音频质量和疾病严重程度进行进一步的亚组分析。使用基于变压器的强制对齐提取的特征与黄金标准具有最高的一致性，包括在音频质量和疾病严重程度方面。这项研究为未来临床医生的智能诊断支持系统和在家远程跟踪疾病进展的新方法奠定了基础。摘要：The analysis of speech measures in individuals with amyotrophic lateral sclerosis (ALS) can provide essential information for early diagnosis and tracking disease progression. However, current methods for extracting speech and pause features are manual or semi-automatic, which makes them time-consuming and labour-intensive. The advent of speech-text alignment algorithms provides an opportunity for inexpensive, automated, and accurate analysis of speech measures in individuals with ALS. There is a need to validate speech and pause features calculated by these algorithms against current gold standard methods. In this study, we extracted 8 speech/pause features from 646 audio files of individuals with ALS and healthy controls performing passage reading. Two pretrained forced alignment models - one using transformers and another using a Gaussian mixture / hidden Markov architecture - were used for automatic feature extraction. The results were then validated against semi-automatic speech/pause analysis software, with further subgroup analyses based on audio quality and disease severity. Features extracted using transformer-based forced alignment had the highest agreement with gold standards, including in terms of audio quality and disease severity. This study lays the groundwork for future intelligent diagnostic support systems for clinicians, and for novel methods of tracking disease progression remotely from home.
【3】 DualVoice: Speech Interaction that Discriminates between Normal and Whispered Voice Input
标题：DualVoice：区分正常语音输入和耳语语音输入的语音交互
链接：https://arxiv.org/abs/2208.10499
机构：The University of Tokyo, Bunkyo-ku, Tokyo, Japan, Sony CSL Kyoto, Kyoto-shi, Kyoto, Japan备注：to appear as ACM UIST 2022 paper摘要：基于自动语音识别（ASR）的交互已经被广泛使用，语音输入越来越多地被用于创建文档。然而，由于没有容易的方法来区分正在发出的命令和需要以语音输入的文本，因此难以识别和校正误识别，这意味着需要手动编辑和校正文档。符号和命令的输入也是具有挑战性的，因为这些可能被误识别为文本字母。为了解决这些问题，本研究提出了一种称为DualVoice的语音交互方法，通过该方法可以用耳语输入命令，用正常语音输入字母。所提出的方法除了常规麦克风之外不需要任何专门的硬件，从而实现完全的免提交互。该方法可用于其中语音识别已经可用的广泛范围的情况，从文本输入到移动/可穿戴计算。本研究设计了两个神经网络，一个用于正常语音和耳语语音的识别，另一个用于耳语语音的识别。然后开发了一个文本输入系统的原型，以显示如何在语音文本输入中使用正常和耳语语音。还讨论了使用DualVoice的其他潜在应用。摘要：Interactions based on automatic speech recognition (ASR) have become widely used, with speech input being increasingly utilized to create documents. However, as there is no easy way to distinguish between commands being issued and text required to be input in speech, misrecognitions are difficult to identify and correct, meaning that documents need to be manually edited and corrected. The input of symbols and commands is also challenging because these may be misrecognized as text letters. To address these problems, this study proposes a speech interaction method called DualVoice, by which commands can be input in a whispered voice and letters in a normal voice. The proposed method does not require any specialized hardware other than a regular microphone, enabling a complete hands-free interaction. The method can be used in a wide range of situations where speech recognition is already available, ranging from text input to mobile/wearable computing. Two neural networks were designed in this study, one for discriminating normal speech from whispered speech, and the second for recognizing whisper speech. A prototype of a text input system was then developed to show how normal and whispered voice can be used in speech text input. Other potential applications using DualVoice are also discussed.
【4】 Are disentangled representations all you need to build speaker anonymization systems?
标题：要建立说话人匿名系统，解开纠缠的表示法就足够了吗？
链接：https://arxiv.org/abs/2208.10497
作者：Pierre Champion,Denis Jouvet,Anthony Larcher机构：Universit´e de Lorraine, CNRS, Inria, LORIA, F-, Nancy, France, LIUM, Le Mans Universit´e, Avenue Olivier Messiaen, LE MANS CEDEX , France摘要：语音信号包含许多敏感信息，例如说话人的身份，这在收集语音数据时引起了隐私问题。说话人匿名化是指在不改变说话内容的前提下，对语音信号进行变换，去除说话人的身份信息。当前的方法通过依赖于内容/说话者解纠缠和语音转换来执行转换。通常，来自自动语音识别系统的声学模型提取内容表示，而x向量系统提取说话者表示。先前的工作已经表明，所提取的特征没有被完美地解缠。本文主要研究如何改进特征解纠缠，从而得到转换后的匿名语音。我们提出通过使用矢量量化从声学模型中去除说话人信息来增强解纠缠。使用VoicePrivacy 2022工具包进行的评估表明，矢量量化有助于隐藏原始说话者身份，同时保持语音识别的实用性。摘要：Speech signals contain a lot of sensitive information, such as the speaker's identity, which raises privacy concerns when speech data get collected. Speaker anonymization aims to transform a speech signal to remove the source speaker's identity while leaving the spoken content unchanged. Current methods perform the transformation by relying on content/speaker disentanglement and voice conversion. Usually, an acoustic model from an automatic speech recognition system extracts the content representation while an x-vector system extracts the speaker representation. Prior work has shown that the extracted features are not perfectly disentangled. This paper tackles how to improve features disentanglement, and thus the converted anonymized speech. We propose enhancing the disentanglement by removing speaker information from the acoustic model using vector quantization. Evaluation done using the VoicePrivacy 2022 toolkit showed that vector quantization helps conceal the original speaker identity while maintaining utility for speech recognition.
【5】 Improving Speech Emotion Recognition Through Focus and Calibration Attention Mechanisms
标题：通过聚焦和校正注意机制改善语音情感识别
链接：https://arxiv.org/abs/2208.10491
作者：Junghun Kim,Yoojin An,Jihie Kim机构：Department of Artificial Intelligence, Dongguk University, Seoul, Korea备注：Accepted by INTERSPEECH 2022摘要：注意力已经成为深度学习方法中最常用的机制之一。注意机制可以帮助系统更多地关注特征空间的关键区域。例如，高幅度区域对于语音情感识别（SER）可以起重要作用。在现有的多人自注意模型中，我们发现了注意力与信号幅度之间的偏差。为了改善注意区域，我们提出了一种聚焦注意（FA）机制和一种新的校准注意（CA）机制，并结合多头部自注意机制。通过FA机制，网络可以检测到片段中幅度最大的部分。通过使用CA机制，网络可以通过为每个注意头分配不同的权重来调整信息流，并提高周围上下文的利用率。为了验证所提方法的有效性，在IEMOCAP和RAVDESS数据集上进行了实验.在两个数据集上的实验结果表明，该框架的性能明显优于现有的方法.摘要：Attention has become one of the most commonly used mechanisms in deep learning approaches. The attention mechanism can help the system focus more on the feature space's critical regions. For example, high amplitude regions can play an important role for Speech Emotion Recognition (SER). In this paper, we identify misalignments between the attention and the signal amplitude in the existing multi-head self-attention. To improve the attention area, we propose to use a Focus-Attention (FA) mechanism and a novel Calibration-Attention (CA) mechanism in combination with the multi-head self-attention. Through the FA mechanism, the network can detect the largest amplitude part in the segment. By employing the CA mechanism, the network can modulate the information flow by assigning different weights to each attention head and improve the utilization of surrounding contexts. To evaluate the proposed method, experiments are performed with the IEMOCAP and RAVDESS datasets. Experimental results show that the proposed framework significantly outperforms the state-of-the-art approaches on both datasets.
【6】 System Fingerprints Detection for DeepFake Audio: An Initial Dataset and Investigation
标题：DeepFake音频的系统指纹检测：初始数据集和研究
链接：https://arxiv.org/abs/2208.10489
作者：Xinrui Yan,Jiangyan Yi,Jianhua Tao,Chenglong Wang,Haoxin Ma,Zhengkun Tian,Ruibo Fu机构：Institute of Automation, Chinese, University of Chinese Academy of, Beijing, China, University of Science and Technology, of China, NLPR, Institute of Automation, Chinese Academy of Sciences, China备注：9 pages, 3 figures. arXiv admin note: text overlap with arXiv:2208.09646摘要：对于deepfake音频检测已经进行了许多有效的尝试。不过，他们也只能分辨真假。对于许多实际应用场景，还需要使用什么工具或算法来生成deepfake音频。这就提出了一个问题：我们能否检测到deepfake audio的系统指纹？因此，本文对检测deepfake音频的系统指纹进行了初步的探讨。在5个最新的深度学习语音合成系统的deepfake音频数据集上进行了实验。结果表明，LFCC特征相对更适合系统指纹检测。此外，ResNet在LCNN和基于x向量的模型中获得了最好的检测结果。t-SNE可视化显示不同的语音合成系统产生不同的系统指纹。摘要：Many effective attempts have been made for deepfake audio detection. However, they can only distinguish between real and fake. For many practical application scenarios, what tool or algorithm generated the deepfake audio also is needed. This raises a question: Can we detect the system fingerprints of deepfake audio? Therefore, this paper conducts a preliminary investigation to detect system fingerprints of deepfake audio. Experiments are conducted on deepfake audio datasets from five latest deep-learning speech synthesis systems. The results show that LFCC features are relatively more suitable for system fingerprints detection. Moreover, the ResNet achieves the best detection results among LCNN and x-vector based models. The t-SNE visualization shows that different speech synthesis systems generate distinct system fingerprints.
【7】 Parameter Sensitivity of Deep-Feature based Evaluation Metrics for Audio Textures
标题：基于深度特征的音频纹理评价指标的参数敏感性
链接：https://arxiv.org/abs/2208.10743
作者：Chitralekha Gupta,Yize Wei,Zequn Gong,Purnima Kamath,Zhuoyao Li,Lonce Wyse机构：National University of Singapore备注：accepted for publication at ISMIR 2022摘要：标准评估度量（如“起始分数”和“Fr\'echet Audio Distance”）提供了合成音频和参考干净音频之间的一般音频质量距离度量。然而，没有很好地研究这些度量对定义音频纹理的统计参数的变化的敏感性。在这项工作中，我们提供了一个系统的研究，一些现有的音频质量评价指标的敏感性，在音频纹理的参数变化。此外，我们还研究了用于音频纹理合成的三个更潜在的参数敏感度量，（a）基于距离的Gram矩阵，（b）使用Gram矩阵的汇总版本的累积Gram度量，以及（c）基于耳蜗模型的统计特征度量。这些度量使用概括任何给定音频纹理的统计的深层特征，因此固有地对定义音频纹理的统计参数的变化敏感。我们研究和评估了现有标准度量以及基于Gram矩阵和耳蜗模型的度量对音频纹理中控制参数变化的敏感性，这些度量跨越了广泛的纹理和参数类型，并通过主观评估进行了验证。我们发现每个度量对不同的纹理参数类型集合敏感。这是研究用于评估音频纹理中的参数敏感性的客观度量的第一步。摘要：Standard evaluation metrics such as the Inception score and Fr\'echet Audio Distance provide a general audio quality distance metric between the synthesized audio and reference clean audio. However, the sensitivity of these metrics to variations in the statistical parameters that define an audio texture is not well studied. In this work, we provide a systematic study of the sensitivity of some of the existing audio quality evaluation metrics to parameter variations in audio textures. Furthermore, we also study three more potentially parameter-sensitive metrics for audio texture synthesis, (a) a Gram matrix based distance, (b) an Accumulated Gram metric using a summarized version of the Gram matrices, and (c) a cochlear-model based statistical features metric. These metrics use deep features that summarize the statistics of any given audio texture, thus being inherently sensitive to variations in the statistical parameters that define an audio texture. We study and evaluate the sensitivity of existing standard metrics as well as Gram matrix and cochlear-model based metrics to control-parameter variations in audio textures across a wide range of texture and parameter types, and validate with subjective evaluation. We find that each of the metrics is sensitive to different sets of texture-parameter types. This is the first step towards investigating objective metrics for assessing parameter sensitivity in audio textures.【1】 Parameter Sensitivity of Deep-Feature based Evaluation Metrics for Audio Textures
标题：基于深度特征的音频纹理评价指标的参数敏感性
链接：https://arxiv.org/abs/2208.10743
* 与cs.SD语音【7】为同一篇
作者：Chitralekha Gupta,Yize Wei,Zequn Gong,Purnima Kamath,Zhuoyao Li,Lonce Wyse机构：National University of Singapore备注：accepted for publication at ISMIR 2022摘要：标准评估度量（如“起始分数”和“Fr\'echet Audio Distance”）提供了合成音频和参考干净音频之间的一般音频质量距离度量。然而，没有很好地研究这些度量对定义音频纹理的统计参数的变化的敏感性。在这项工作中，我们提供了一个系统的研究，一些现有的音频质量评价指标的敏感性，在音频纹理的参数变化。此外，我们还研究了用于音频纹理合成的三个更潜在的参数敏感度量，（a）基于距离的Gram矩阵，（b）使用Gram矩阵的汇总版本的累积Gram度量，以及（c）基于耳蜗模型的统计特征度量。这些度量使用概括任何给定音频纹理的统计的深层特征，因此固有地对定义音频纹理的统计参数的变化敏感。我们研究和评估了现有标准度量以及基于Gram矩阵和耳蜗模型的度量对音频纹理中控制参数变化的敏感性，这些度量跨越了广泛的纹理和参数类型，并通过主观评估进行了验证。我们发现每个度量对不同的纹理参数类型集合敏感。这是研究用于评估音频纹理中的参数敏感性的客观度量的第一步。摘要：Standard evaluation metrics such as the Inception score and Fr\'echet Audio Distance provide a general audio quality distance metric between the synthesized audio and reference clean audio. However, the sensitivity of these metrics to variations in the statistical parameters that define an audio texture is not well studied. In this work, we provide a systematic study of the sensitivity of some of the existing audio quality evaluation metrics to parameter variations in audio textures. Furthermore, we also study three more potentially parameter-sensitive metrics for audio texture synthesis, (a) a Gram matrix based distance, (b) an Accumulated Gram metric using a summarized version of the Gram matrices, and (c) a cochlear-model based statistical features metric. These metrics use deep features that summarize the statistics of any given audio texture, thus being inherently sensitive to variations in the statistical parameters that define an audio texture. We study and evaluate the sensitivity of existing standard metrics as well as Gram matrix and cochlear-model based metrics to control-parameter variations in audio textures across a wide range of texture and parameter types, and validate with subjective evaluation. We find that each of the metrics is sensitive to different sets of texture-parameter types. This is the first step towards investigating objective metrics for assessing parameter sensitivity in audio textures.
【2】 StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation
标题：StyleTalker：基于一次性风格的音频驱动的对讲机头部视频生成
链接：https://arxiv.org/abs/2208.10922
作者：Dongchan Min,Minyoung Song,Sung Ju Hwang机构：Graduate School of AI, Korea Advanced Institute of Science and Technology, (KAIST), Seoul, South Korea, AITRICS, Seoul, South Korea摘要：我们提出了一种新颖的音频驱动的说话人头部生成模型StyleTalker，它可以从单个参考图像中合成说话人的视频，并具有精确的音频同步唇形、逼真的头部姿势和眨眼。具体地，通过利用预先训练的图像生成器和图像编码器，我们估计忠实地反映给定音频的正在说话的头部视频的潜在代码。这是通过几个新设计的组件实现的：1）用于精确唇同步的对比唇同步鉴别器，2）条件顺序变分自动编码器，其学习从唇运动中分离的潜在运动空间，使得我们可以独立地操纵运动和唇运动，同时保持身份。3)用归一化流扩充的自回归先验，以学习复杂的音频到运动多模态潜在空间。配备了这些组件，StyleTalker不仅可以在提供另一个运动源视频时以运动可控的方式生成会说话的头部视频，还可以通过从输入音频推断逼真的运动，以完全音频驱动的方式生成。通过大量的实验和用户研究，我们证明了我们的模型能够合成具有令人印象深刻的感知质量的说话的头部视频，这些视频与输入音频精确地唇形同步，大大超过了最先进的基线。摘要：We propose StyleTalker, a novel audio-driven talking head generation model that can synthesize a video of a talking person from a single reference image with accurately audio-synced lip shapes, realistic head poses, and eye blinks. Specifically, by leveraging a pretrained image generator and an image encoder, we estimate the latent codes of the talking head video that faithfully reflects the given audio. This is made possible with several newly devised components: 1) A contrastive lip-sync discriminator for accurate lip synchronization, 2) A conditional sequential variational autoencoder that learns the latent motion space disentangled from the lip movements, such that we can independently manipulate the motions and lip movements while preserving the identity. 3) An auto-regressive prior augmented with normalizing flow to learn a complex audio-to-motion multi-modal latent space. Equipped with these components, StyleTalker can generate talking head videos not only in a motion-controllable way when another motion source video is given but also in a completely audio-driven manner by inferring realistic motions from the input audio. Through extensive experiments and user studies, we show that our model is able to synthesize talking head videos with impressive perceptual quality which are accurately lip-synced with the input audios, largely outperforming state-of-the-art baselines.
【3】 In-Air Imaging Sonar Sensor Network with Real-Time Processing Using GPUs
标题：利用GPU实现实时处理的空中成像声纳传感器网络
链接：https://arxiv.org/abs/2208.10839
作者：Wouter Jansen,Dennis Laurijssen,Robin Kerstens,Walter Daems,Jan Steckel机构：Jan SteckelUniversity of Antwerp备注：2019 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing摘要：对于自主导航和机器人应用，正确感知环境至关重要。存在用于此目的的许多感测模态。近年来，正在使用的一种这样的模态是空中成像声纳。它是灰尘或雾等恶劣条件下的复杂环境的理想选择。然而，与大多数感测模态一样，为了感测移动平台周围的全部环境，需要多个这样的传感器来捕捉全部360度范围。目前，用于创建该数据的处理算法不足以以合理快速的更新速率为多个传感器创建该数据。此外，需要一个灵活和鲁棒的框架，以容易地将多个成像声纳传感器实施到任何设置中，并为数据的多种应用类型服务。在本文中，我们提出了一个传感器网络框架设计，为这种新的传感模态。此外，提出了在图形处理单元上实现处理算法，以潜在地减少计算时间，从而允许以足够高的更新速率实时处理一个或多个成像声纳传感器。摘要：For autonomous navigation and robotic applications, sensing the environment correctly is crucial. Many sensing modalities for this purpose exist. In recent years, one such modality that is being used is in-air imaging sonar. It is ideal in complex environments with rough conditions such as dust or fog. However, like with most sensing modalities, to sense the full environment around the mobile platform, multiple such sensors are needed to capture the full 360-degree range. Currently the processing algorithms used to create this data are insufficient to do so for multiple sensors at a reasonably fast update rate. Furthermore, a flexible and robust framework is needed to easily implement multiple imaging sonar sensors into any setup and serve multiple application types for the data. In this paper we present a sensor network framework designed for this novel sensing modality. Furthermore, an implementation of the processing algorithm on a Graphics Processing Unit is proposed to potentially decrease the computing time to allow for real-time processing of one or more imaging sonar sensors at a sufficiently high update rate.
【4】 Fall Detection from Audios with Audio Transformers
标题：使用Audio Transformers从音频中检测跌倒
链接：https://arxiv.org/abs/2208.10659
* 与cs.SD语音【1】为同一篇
作者：Prabhjot Kaur,Qifan Wang,Weisong Shi机构：Wayne State University, Detroit, Michigan, USA, Meta AI, Menlo Park, California, USA摘要：针对老年人的跌倒检测是一个经过充分研究的问题，提出了几种解决方案，包括可穿戴和非可穿戴技术。虽然现有技术具有极好的检测率，但是由于需要佩戴设备和用户隐私问题，它们缺乏目标人群的采用。本文提出了一种新颖的、非穿戴式、非侵入式、可扩展的跌倒检测解决方案，部署在一个带有麦克风的自主移动机器人上。所提出的方法使用在人们家中记录的环境声音输入。我们特别针对浴室环境，因为它非常容易跌倒，并且现有技术无法在不危及用户隐私的情况下部署。本工作开发了Transformer基于Transformer架构的解决方案，该架构从浴室获取噪声声音输入，并将其分类为落音/非落音类，准确度为0.8673。此外，所提出的方法可扩展到除浴室之外的其它室内环境，并且适合于部署在养老院、医院和康复机构中，而不需要用户佩戴任何设备或由传感器持续“监视”。摘要：Fall detection for the elderly is a well-researched problem with several proposed solutions, including wearable and non-wearable techniques. While the existing techniques have excellent detection rates, their adoption by the target population is lacking due to the need for wearing devices and user privacy concerns. Our paper provides a novel, non-wearable, non-intrusive, and scalable solution for fall detection, deployed on an autonomous mobile robot equipped with a microphone. The proposed method uses ambient sound input recorded in people's homes. We specifically target the bathroom environment as it is highly prone to falls and where existing techniques cannot be deployed without jeopardizing user privacy. The present work develops a solution based on a Transformer architecture that takes noisy sound input from bathrooms and classifies it into fall/no-fall class with an accuracy of 0.8673. Further, the proposed approach is extendable to other indoor environments, besides bathrooms and is suitable for deploying in elderly homes, hospitals, and rehabilitation facilities without requiring the user to wear any device or be constantly "watched" by the sensors.
【5】 Concurrent Validity of Automatic Speech and Pause Measures During Passage Reading in ALS标题：肌萎缩侧索硬化症患者短文阅读中自动语音和停顿测量的同时效度
链接：https://arxiv.org/abs/2208.10597
* 与cs.SD语音【2】为同一篇
作者：Saeid Alavi Naeini,Leif Simmatis,Yana Yunusova,Babak Taati机构：Kite Research Institute, Toronto Rehabilitation Institute – University Health Network, Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada, Department of Speech-Language Pathology, University of Toronto, Toronto, Canada备注：2022 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI)摘要：分析肌萎缩侧索硬化症（ALS）患者的语音测量值可以为早期诊断和跟踪疾病进展提供必要的信息。然而，目前用于提取语音和停顿特征的方法是手动或半自动的，这使得它们耗时且劳动强度大。语音—文本对齐算法的出现为ALS患者的语音测量提供了廉价、自动和准确的分析机会。需要对照当前的黄金标准方法来验证由这些算法计算的语音和停顿特征。在本研究中，我们从646个ALS患者和健康对照者进行段落阅读的音频文件中提取了8个语音/停顿特征。两个预先训练的强制对齐模型—一个使用Transformers，另一个使用高斯混合/隐马尔可夫结构—用于自动特征提取。然后用半自动语音/停顿分析软件对结果进行验证，并根据音频质量和疾病严重程度进行进一步的亚组分析。使用基于变压器的强制对齐提取的特征与黄金标准具有最高的一致性，包括在音频质量和疾病严重程度方面。这项研究为未来临床医生的智能诊断支持系统和在家远程跟踪疾病进展的新方法奠定了基础。摘要：The analysis of speech measures in individuals with amyotrophic lateral sclerosis (ALS) can provide essential information for early diagnosis and tracking disease progression. However, current methods for extracting speech and pause features are manual or semi-automatic, which makes them time-consuming and labour-intensive. The advent of speech-text alignment algorithms provides an opportunity for inexpensive, automated, and accurate analysis of speech measures in individuals with ALS. There is a need to validate speech and pause features calculated by these algorithms against current gold standard methods. In this study, we extracted 8 speech/pause features from 646 audio files of individuals with ALS and healthy controls performing passage reading. Two pretrained forced alignment models - one using transformers and another using a Gaussian mixture / hidden Markov architecture - were used for automatic feature extraction. The results were then validated against semi-automatic speech/pause analysis software, with further subgroup analyses based on audio quality and disease severity. Features extracted using transformer-based forced alignment had the highest agreement with gold standards, including in terms of audio quality and disease severity. This study lays the groundwork for future intelligent diagnostic support systems for clinicians, and for novel methods of tracking disease progression remotely from home.
【6】 DualVoice: Speech Interaction that Discriminates between Normal and Whispered Voice Input
标题：DualVoice：区分正常语音输入和耳语语音输入的语音交互
链接：https://arxiv.org/abs/2208.10499
* 与cs.SD语音【3】为同一篇
机构：The University of Tokyo, Bunkyo-ku, Tokyo, Japan, Sony CSL Kyoto, Kyoto-shi, Kyoto, Japan备注：to appear as ACM UIST 2022 paper摘要：基于自动语音识别（ASR）的交互已经被广泛使用，语音输入越来越多地被用于创建文档。然而，由于没有容易的方法来区分正在发出的命令和需要以语音输入的文本，因此难以识别和校正误识别，这意味着需要手动编辑和校正文档。符号和命令的输入也是具有挑战性的，因为这些可能被误识别为文本字母。为了解决这些问题，本研究提出了一种称为DualVoice的语音交互方法，通过该方法可以用耳语输入命令，用正常语音输入字母。所提出的方法除了常规麦克风之外不需要任何专门的硬件，从而实现完全的免提交互。该方法可用于其中语音识别已经可用的广泛范围的情况，从文本输入到移动/可穿戴计算。本研究设计了两个神经网络，一个用于正常语音和耳语语音的识别，另一个用于耳语语音的识别。然后开发了一个文本输入系统的原型，以显示如何在语音文本输入中使用正常和耳语语音。还讨论了使用DualVoice的其他潜在应用。摘要：Interactions based on automatic speech recognition (ASR) have become widely used, with speech input being increasingly utilized to create documents. However, as there is no easy way to distinguish between commands being issued and text required to be input in speech, misrecognitions are difficult to identify and correct, meaning that documents need to be manually edited and corrected. The input of symbols and commands is also challenging because these may be misrecognized as text letters. To address these problems, this study proposes a speech interaction method called DualVoice, by which commands can be input in a whispered voice and letters in a normal voice. The proposed method does not require any specialized hardware other than a regular microphone, enabling a complete hands-free interaction. The method can be used in a wide range of situations where speech recognition is already available, ranging from text input to mobile/wearable computing. Two neural networks were designed in this study, one for discriminating normal speech from whispered speech, and the second for recognizing whisper speech. A prototype of a text input system was then developed to show how normal and whispered voice can be used in speech text input. Other potential applications using DualVoice are also discussed.
【7】 Are disentangled representations all you need to build speaker anonymization systems?
标题：要建立说话人匿名系统，解开纠缠的表示法就足够了吗？
链接：https://arxiv.org/abs/2208.10497
* 与cs.SD语音【4】为同一篇
作者：Pierre Champion,Denis Jouvet,Anthony Larcher机构：Universit´e de Lorraine, CNRS, Inria, LORIA, F-, Nancy, France, LIUM, Le Mans Universit´e, Avenue Olivier Messiaen, LE MANS CEDEX , France摘要：语音信号包含许多敏感信息，例如说话人的身份，这在收集语音数据时引起了隐私问题。说话人匿名化是指在不改变说话内容的前提下，对语音信号进行变换，去除说话人的身份信息。当前的方法通过依赖于内容/说话者解纠缠和语音转换来执行转换。通常，来自自动语音识别系统的声学模型提取内容表示，而x向量系统提取说话者表示。先前的工作已经表明，所提取的特征没有被完美地解缠。本文主要研究如何改进特征解纠缠，从而得到转换后的匿名语音。我们提出通过使用矢量量化从声学模型中去除说话人信息来增强解纠缠。使用VoicePrivacy 2022工具包进行的评估表明，矢量量化有助于隐藏原始说话者身份，同时保持语音识别的实用性。摘要：Speech signals contain a lot of sensitive information, such as the speaker's identity, which raises privacy concerns when speech data get collected. Speaker anonymization aims to transform a speech signal to remove the source speaker's identity while leaving the spoken content unchanged. Current methods perform the transformation by relying on content/speaker disentanglement and voice conversion. Usually, an acoustic model from an automatic speech recognition system extracts the content representation while an x-vector system extracts the speaker representation. Prior work has shown that the extracted features are not perfectly disentangled. This paper tackles how to improve features disentanglement, and thus the converted anonymized speech. We propose enhancing the disentanglement by removing speaker information from the acoustic model using vector quantization. Evaluation done using the VoicePrivacy 2022 toolkit showed that vector quantization helps conceal the original speaker identity while maintaining utility for speech recognition.
【8】 Improving Speech Emotion Recognition Through Focus and Calibration Attention Mechanisms
标题：通过聚焦和校正注意机制改善语音情感识别
链接：https://arxiv.org/abs/2208.10491
* 与cs.SD语音【5】为同一篇
作者：Junghun Kim,Yoojin An,Jihie Kim机构：Department of Artificial Intelligence, Dongguk University, Seoul, Korea备注：Accepted by INTERSPEECH 2022摘要：注意力已经成为深度学习方法中最常用的机制之一。注意机制可以帮助系统更多地关注特征空间的关键区域。例如，高幅度区域对于语音情感识别（SER）可以起重要作用。在现有的多人自注意模型中，我们发现了注意力与信号幅度之间的偏差。为了改善注意区域，我们提出了一种聚焦注意（FA）机制和一种新的校准注意（CA）机制，并结合多头部自注意机制。通过FA机制，网络可以检测到片段中幅度最大的部分。通过使用CA机制，网络可以通过为每个注意头分配不同的权重来调整信息流，并提高周围上下文的利用率。为了验证所提方法的有效性，在IEMOCAP和RAVDESS数据集上进行了实验.在两个数据集上的实验结果表明，该框架的性能明显优于现有的方法.摘要：Attention has become one of the most commonly used mechanisms in deep learning approaches. The attention mechanism can help the system focus more on the feature space's critical regions. For example, high amplitude regions can play an important role for Speech Emotion Recognition (SER). In this paper, we identify misalignments between the attention and the signal amplitude in the existing multi-head self-attention. To improve the attention area, we propose to use a Focus-Attention (FA) mechanism and a novel Calibration-Attention (CA) mechanism in combination with the multi-head self-attention. Through the FA mechanism, the network can detect the largest amplitude part in the segment. By employing the CA mechanism, the network can modulate the information flow by assigning different weights to each attention head and improve the utilization of surrounding contexts. To evaluate the proposed method, experiments are performed with the IEMOCAP and RAVDESS datasets. Experimental results show that the proposed framework significantly outperforms the state-of-the-art approaches on both datasets.
【9】 System Fingerprints Detection for DeepFake Audio: An Initial Dataset and Investigation
标题：DeepFake音频的系统指纹检测：初始数据集和研究
链接：https://arxiv.org/abs/2208.10489
* 与cs.SD语音【6】为同一篇
作者：Xinrui Yan,Jiangyan Yi,Jianhua Tao,Chenglong Wang,Haoxin Ma,Zhengkun Tian,Ruibo Fu机构：Institute of Automation, Chinese, University of Chinese Academy of, Beijing, China, University of Science and Technology, of China, NLPR, Institute of Automation, Chinese Academy of Sciences, China备注：9 pages, 3 figures. arXiv admin note: text overlap with arXiv:2208.09646摘要：对于deepfake音频检测已经进行了许多有效的尝试。不过，他们也只能分辨真假。对于许多实际应用场景，还需要使用什么工具或算法来生成deepfake音频。这就提出了一个问题：我们能否检测到deepfake audio的系统指纹？因此，本文对检测deepfake音频的系统指纹进行了初步的探讨。在5个最新的深度学习语音合成系统的deepfake音频数据集上进行了实验。结果表明，LFCC特征相对更适合系统指纹检测。此外，ResNet在LCNN和基于x向量的模型中获得了最好的检测结果。t-SNE可视化显示不同的语音合成系统产生不同的系统指纹。摘要：Many effective attempts have been made for deepfake audio detection. However, they can only distinguish between real and fake. For many practical application scenarios, what tool or algorithm generated the deepfake audio also is needed. This raises a question: Can we detect the system fingerprints of deepfake audio? Therefore, this paper conducts a preliminary investigation to detect system fingerprints of deepfake audio. Experiments are conducted on deepfake audio datasets from five latest deep-learning speech synthesis systems. The results show that LFCC features are relatively more suitable for system fingerprints detection. Moreover, the ResNet achieves the best detection results among LCNN and x-vector based models. The t-SNE visualization shows that different speech synthesis systems generate distinct system fingerprints.
机器翻译，仅供参考
每日语音速递