今日论文合集：cs.SD语音22篇，eess.AS音频处理24篇。【1】 Mono-to-stereo through parametric stereo generation标题：通过参数立体声生成实现单声道到立体声链接：https://arxiv.org/abs/2306.14647作者：Joan Serrà,Davide Scaini,Santiago Pascual,Daniel Arteaga,Jordi Pons,Jeroen Breebaart,Giulio Cengarle机构：Dolby Laboratories备注：7 pages, 1 figure; accepted for ISMIR23摘要：从单声道音频信号生成立体声呈现是一项具有挑战性的开放式任务，特别是如果目标是获得具有声音元素的特定平移的逼真空间成像。在这项工作中，我们建议通过使用最近邻和深度网络方法预测参数立体声（PS）参数来将单声道转换为立体声。结合PS，我们还提出建模的任务生成方法，允许合成多个和同样合理的立体声再现从相同的单声道信号。为了实现这一目标，我们考虑了自回归和屏蔽令牌建模方法。我们提供的证据表明，拟议的基于PS的模型优于竞争力的经典去相关基线，在PS预测框架内，现代生成模型胜过同等的非生成模型。总的来说，我们的工作定位PS和生成建模作为单声道到立体声上混的强大和有吸引力的方法。还提供了这些方法的局限性的讨论。摘要：Generating a stereophonic presentation from a monophonic audio signal is a challenging open task, especially if the goal is to obtain a realistic spatial imaging with a specific panning of sound elements. In this work, we propose to convert mono to stereo by means of predicting parametric stereo (PS) parameters using both nearest neighbor and deep network approaches. In combination with PS, we also propose to model the task with generative approaches, allowing to synthesize multiple and equally-plausible stereo renditions from the same mono signal. To achieve this, we consider both autoregressive and masked token modelling approaches. We provide evidence that the proposed PS-based models outperform a competitive classical decorrelation baseline and that, within a PS prediction framework, modern generative models outshine equivalent non-generative counterparts. Overall, our work positions both PS and generative modelling as strong and appealing methodologies for mono-to-stereo upmixing. A discussion of the limitations of these approaches is also provided.【2】 Cross-Lingual Cross-Age Group Adaptation for Low-Resource Elderly Speech Emotion Recognition标题：低资源老年人语音情感识别的跨语言跨年龄组自适应链接：https://arxiv.org/abs/2306.14517作者：Samuel Cahyawijaya,Holy Lovenia,Willy Chung,Rita Frieske,Zihan Liu,Pascale Fung机构：The Hong Kong University of Science and Technology备注：Accepted in INTERSPEECH 2023摘要：语音情感识别在人机交互中起着至关重要的作用。然而，大多数语音情感识别研究都偏向于讲英语的成年人，这阻碍了其适用于不同语言和年龄段的其他人口统计群体。在这项工作中，我们分析了情绪识别在三种不同语言之间的可转移性-英语，汉语普通话和广东话;和两个不同的年龄组成年人和老年人。为了进行实验，我们开发了一个成年人和老年人的英语-普通话语音情感基准，BiMotion，和粤语语音情感数据集，YueMotion。本研究的结论是，不同的语言和年龄组需要特定的语音特征，从而使跨语言推理的方法是不合适的。然而，跨组数据增强仍然有利于正则化模型，语言距离对跨语言可移植性有重要影响。我们在www.example.com公开发布我们的代码https://github.com/HLTCHKUST/elderly_ser。摘要：Speech emotion recognition plays a crucial role in human-computer interactions. However, most speech emotion recognition research is biased toward English-speaking adults, which hinders its applicability to other demographic groups in different languages and age groups. In this work, we analyze the transferability of emotion recognition across three different languages--English, Mandarin Chinese, and Cantonese; and 2 different age groups--adults and the elderly. To conduct the experiment, we develop an English-Mandarin speech emotion benchmark for adults and the elderly, BiMotion, and a Cantonese speech emotion dataset, YueMotion. This study concludes that different language and age groups require specific speech features, thus making cross-lingual inference an unsuitable method. However, cross-group data augmentation is still beneficial to regularize the model, with linguistic distance being a significant influence on cross-lingual transferability. We release publicly release our code at https://github.com/HLTCHKUST/elderly_ser.【3】 The Singing Voice Conversion Challenge 2023标题：2023年歌声转换挑战赛链接：https://arxiv.org/abs/2306.14422作者：Wen-Chin Huang,Lester Phillip Violeta,Songxiang Liu,Jiatong Shi,Yusuke Yasuda,Tomoki Toda机构：Nagoya University, Japan, Tencent AI Lab, Carnegie Mellon University, USA摘要：我们介绍了语音转换挑战（VCC）系列的最新版本，这是一项每两年举办一次的科学活动，旨在比较和了解基于通用数据集的不同语音转换（VC）系统。今年，我们将重点转移到歌唱声音转换（SVC），因此将挑战命名为歌唱声音转换挑战（SVCC）。构建了一个新的数据库，用于两个任务，即域内和跨域SVC。挑战赛进行了两个月，我们总共收到了26份提交，包括2个基线。通过大规模的众包听力测试，我们观察到，对于这两个任务，虽然人类水平的自然度是由顶级系统实现的，但没有一个团队能够获得与目标说话者一样高的相似度分数。此外，如预期的，跨域SVC比域内SVC更难，特别是在相似性方面。我们还调查了现有的客观测量是否能够预测感知性能，并发现只有少数人可以达到显着的相关性。摘要：We present the latest iteration of the voice conversion challenge (VCC) series, a bi-annual scientific event aiming to compare and understand different voice conversion (VC) systems based on a common dataset. This year we shifted our focus to singing voice conversion (SVC), thus named the challenge the Singing Voice Conversion Challenge (SVCC). A new database was constructed for two tasks, namely in-domain and cross-domain SVC. The challenge was run for two months, and in total we received 26 submissions, including 2 baselines. Through a large-scale crowd-sourced listening test, we observed that for both tasks, although human-level naturalness was achieved by the top system, no team was able to obtain a similarity score as high as the target speakers. Also, as expected, cross-domain SVC is harder than in-domain SVC, especially in the similarity aspect. We also investigated whether existing objective measurements were able to predict perceptual performance, and found that only few of them could reach a significant correlation.【4】 Addressing Cold Start Problem for End-to-end Automatic Speech Scoring标题：解决端到端自动语音评分的冷启动问题链接：https://arxiv.org/abs/2306.14310作者：Jungbae Park,Seungtaek Choi机构：Riiid AI Research备注：Accepted at Interspeech 2023, 4 pages, 1 page for reference摘要：集成自动语音评分/评估系统已成为第二语言口语教育的一个重要方面。随着自我监督学习的进步，端到端语音评分方法已经显示出有希望的结果。然而，这项研究强调了在新的问题背景下，语音评分系统的性能显着下降，从而确定这是一个冷启动问题的项目。随着冷启动现象的发现，本文试图通过以下方法缓解该问题：1）提示嵌入，2）使用BERT或CLIP模型的问题上下文嵌入，以及3）预训练的声学模型的选择。实验是在托业口语测试数据集上进行的，这些数据集是由专业的托业口语评估员对英语作为第二语言（ESL）的学习者进行评估的。结果表明，所提出的框架不仅表现出在冷启动环境中的鲁棒性，但也优于已知内容的基线。摘要：Integrating automatic speech scoring/assessment systems has become a critical aspect of second-language speaking education. With self-supervised learning advancements, end-to-end speech scoring approaches have exhibited promising results. However, this study highlights the significant decrease in the performance of speech scoring systems in new question contexts, thereby identifying this as a cold start problem in terms of items. With the finding of cold-start phenomena, this paper seeks to alleviate the problem by following methods: 1) prompt embeddings, 2) question context embeddings using BERT or CLIP models, and 3) choice of the pretrained acoustic model. Experiments are conducted on TOEIC speaking test datasets collected from English-as-a-second-language (ESL) learners rated by professional TOEIC speaking evaluators. The results demonstrate that the proposed framework not only exhibits robustness in a cold-start environment but also outperforms the baselines for known content.【5】 Aeroacoustic Source Localization标题：气动声源定位链接：https://arxiv.org/abs/2306.14276作者：Weicheng Xue,Bing Yang,Shaohong Jia机构：• This work applied the beamforming algorithm and the DAMAS algo-, rithm to perform sound source localization for numerical simulations, as well as real-world experiments for a ,., MW wind turbine.摘要：解卷积DAMAS算法可以有效地消除通常使用的波束形成定位算法中的误解，允许更准确地计算源位置以及强度。在求解线性方程组时，DAMAS算法考虑了不同位置的相互影响，减少甚至消除了旁瓣，产生了更精确的结果。 本文首先介绍了DAMAS算法的原理。然后，它适用于波束形成算法和DAMAS算法来模拟定位的单频源从1.5兆瓦的风力涡轮机，一个复杂的线源与文本“UCAS”和线源下游的翼型后缘。最后，工作提出了实验定位结果的源的1.5兆瓦的风力涡轮机使用波束形成算法和DAMAS算法。摘要：The deconvolutional DAMAS algorithm can effectively eliminate the misconceptions in the usually-used beamforming localization algorithm, allowing for more accurate calculation of the source location as well as the intensity. When solving a linear system of equations, the DAMAS algorithm takes into account the mutual influence of different locations, reducing or even eliminating sidelobes and producing more accurate results. This work first introduces the principles of the DAMAS algorithm. Then it applies both the beamforming algorithm and the DAMAS algorithm to simulate the localization of a single-frequency source from a 1.5 MW wind turbine, a complex line source with the text "UCAS" and a line source downstream of an airfoil trailing edge. Finally, the work presents experimental localization results of the source of a 1.5 MW wind turbine using both the beamforming algorithm and the DAMAS algorithm.【6】 PrimaDNN': A Characteristics-aware DNN Customization for Singing Technique Detection标题：PrimaDNN‘：一种用于歌唱技巧检测的特征感知DNN定制链接：https://arxiv.org/abs/2306.14191作者：Yuya Yamamoto,Juhan Nam,Hiroko Terasawa机构：Doctoral Program in Informatics, University of Tsukuba, Ibaraki, Japan, Graduate School of Culture Technology, KAIST, Daejeon, South Korea备注：Accepted at EUSIPCO 2023摘要：专业歌手调整他们的声音音色或音高，使他们的声乐表演更具表现力。这种波动被称为歌唱技巧。从音轨中自动检测歌唱技巧可以有益于理解每个歌唱者如何表达表演，然而由于歌唱技巧的广泛多样性，这也可能是困难的。深度神经网络（DNN）模型可以处理这种变化;然而，考虑数据的特性可能会提高歌唱技术检测的性能。在本文中，我们提出了PrimaDNN，这是一种具有面向特征的改进的CRNN模型。该模型的特点是：1）基于辅助音高信息和多分辨率梅尔频谱图的输入特征表示，2）基于挤压和激励（SENet）和实例归一化的卷积模块。在J-POP歌唱技术检测结果中，PrimaDNN在整体macro-F测量中取得了44.9%的最佳结果，与传统作品相比。我们还发现，每个组件的贡献不同，这取决于歌唱技术的类型。摘要：Professional vocalists modulate their voice timbre or pitch to make their vocal performance more expressive. Such fluctuations are called singing techniques. Automatic detection of singing techniques from audio tracks can be beneficial to understand how each singer expresses the performance, yet it can also be difficult due to the wide variety of the singing techniques. A deep neural network (DNN) model can handle such variety; however, there might be a possibility that considering the characteristics of the data improves the performance of singing technique detection. In this paper, we propose PrimaDNN, a CRNN model with a characteristics-oriented improvement. The features of the model are: 1) input feature representation based on auxiliary pitch information and multi-resolution mel spectrograms, 2) Convolution module based on the Squeeze-and-excitation (SENet) and the Instance normalization. In the results of J-POP singing technique detection, PrimaDNN achieved the best results of 44.9% at the overall macro-F measure, compared to conventional works. We also found that the contribution of each component varies depending on the type of singing technique.【7】 AV-SepFormer: Cross-Attention SepFormer for Audio-Visual Target Speaker Extraction标题：用于视听目标说话人提取的交叉注意SepFormer链接：https://arxiv.org/abs/2306.14170作者：Jiuxin Lin,Xinyu Cai,Heinrich Dinkel,Jun Chen,Zhiyong Yan,Yongqing Wang,Junbo Zhang,Zhiyong Wu,Yujun Wang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, Xiaomi Inc., Beijing, China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：Accepted by ICASSP2023摘要：视觉信息可以作为一个有效的线索，目标说话人提取（TSE），是至关重要的，以提高提取性能。在本文中，我们提出了AV-SepFormer，一个基于SepFormer的注意力双尺度模型，利用交叉和自我注意力融合和模型特征的音频和视觉。AV-SepFormer将音频特征分割成多个块，相当于视觉特征的长度。然后利用自注意和交叉注意对多模态特征进行建模和融合。此外，我们使用了一种新的2D位置编码，它引入了块之间和块内的位置信息，并提供了显着的收益超过传统的位置编码。我们的模型有两个关键优势：将音频分块特征的时间粒度与视觉特征同步，缓解了音视频采样率不一致带来的危害;通过组合自注意和交叉注意，特征融合和语音提取过程在注意范例内被统一。实验结果表明，AV-SepFormer算法明显优于现有的其他方法。摘要：Visual information can serve as an effective cue for target speaker extraction (TSE) and is vital to improving extraction performance. In this paper, we propose AV-SepFormer, a SepFormer-based attention dual-scale model that utilizes cross- and self-attention to fuse and model features from audio and visual. AV-SepFormer splits the audio feature into a number of chunks, equivalent to the length of the visual feature. Then self- and cross-attention are employed to model and fuse the multi-modal features. Furthermore, we use a novel 2D positional encoding, that introduces the positional information between and within chunks and provides significant gains over the traditional positional encoding. Our model has two key advantages: the time granularity of audio chunked feature is synchronized to the visual feature, which alleviates the harm caused by the inconsistency of audio and video sampling rate; by combining self- and cross-attention, feature fusion and speech extraction processes are unified within an attention paradigm. The experimental results show that AV-SepFormer significantly outperforms other existing methods.【8】 DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech标题：DSE-TTS：用于跨语言文语转换的双说话人嵌入链接：https://arxiv.org/abs/2306.14145作者：Sen Liu,Yiwei Guo,Chenpeng Du,Xie Chen,Kai Yu机构：MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China备注：Accepted to Interspeech 2023摘要：虽然对于语内语音合成可以获得高保真语音，但是跨语言文本到语音（CTTS）仍然远远不能令人满意，因为难以准确地保留说话者音色（即，语音合成）。说话者相似性）并从他们的第一语言中消除口音（即，原生性）。在本文中，我们证明了矢量量化（VQ）的声学特征包含较少的说话人信息比梅尔频谱图。基于这一发现，我们提出了一种新的双说话人嵌入TTS（DSE-TTS）框架CTTS与真实的说话风格。这里，一个嵌入被馈送到声学模型以学习语言说话风格，而另一个被集成到声码器中以模仿目标说话者的音色。实验表明，通过结合这两种嵌入，DSE-TTS显着优于国家的最先进的SANE-TTS在跨语言合成，特别是在本土化方面。摘要：Although high-fidelity speech can be obtained for intralingual speech synthesis, cross-lingual text-to-speech (CTTS) is still far from satisfactory as it is difficult to accurately retain the speaker timbres(i.e. speaker similarity) and eliminate the accents from their first language(i.e. nativeness). In this paper, we demonstrated that vector-quantized(VQ) acoustic feature contains less speaker information than mel-spectrogram. Based on this finding, we propose a novel dual speaker embedding TTS (DSE-TTS) framework for CTTS with authentic speaking style. Here, one embedding is fed to the acoustic model to learn the linguistic speaking style, while the other one is integrated into the vocoder to mimic the target speaker's timbre. Experiments show that by combining both embeddings, DSE-TTS significantly outperforms the state-of-the-art SANE-TTS in cross-lingual synthesis, especially in terms of nativeness.【9】 An Analysis of Personalized Speech Recognition System Development for the Deaf and Hard-of-Hearing标题：聋哑人个性化语音识别系统开发分析链接：https://arxiv.org/abs/2306.13953作者：Lester Phillip Violeta,Tomoki Toda机构：Nagoya University, Japan备注：Submitted to APSIPA 2023摘要：耳聋或重听（DHH）说话者通常具有由耳聋引起的非典型言语。随着基于语音的设备和软件应用程序的支持不断增加，需要做更多的工作来使这些设备对每个人都具有包容性。要做到这一点，我们分析了开放可用的自动语音识别（ASR）工具与DHH日语说话者数据集的使用。由于这些开箱即用的ASR模型通常不表现良好的DHH语音，我们提供了创建个性化ASR系统的彻底分析。我们收集了一个大的DHH说话者数据集，四个说话者总计约28.05小时，并通过改变训练数据大小彻底分析了不同训练框架的性能。我们的研究结果表明，1000个话语（或1-2小时）从目标说话人已经可以显着提高模型的性能与最少的工作量所需的，因此我们建议研究人员收集至少1000个话语，使一个有效的个性化ASR系统。在1000个话语难以收集的情况下，我们还发现显着的改进，在使用先前提出的数据增强技术，如中间微调时，只有200个话语可用。摘要：Deaf or hard-of-hearing (DHH) speakers typically have atypical speech caused by deafness. With the growing support of speech-based devices and software applications, more work needs to be done to make these devices inclusive to everyone. To do so, we analyze the use of openly-available automatic speech recognition (ASR) tools with a DHH Japanese speaker dataset. As these out-of-the-box ASR models typically do not perform well on DHH speech, we provide a thorough analysis of creating personalized ASR systems. We collected a large DHH speaker dataset of four speakers totaling around 28.05 hours and thoroughly analyzed the performance of different training frameworks by varying the training data sizes. Our findings show that 1000 utterances (or 1-2 hours) from a target speaker can already significantly improve the model performance with minimal amount of work needed, thus we recommend researchers to collect at least 1000 utterances to make an efficient personalized ASR system. In cases where 1000 utterances is difficult to collect, we also discover significant improvements in using previously proposed data augmentation techniques such as intermediate fine-tuning when only 200 utterances are available.【10】 Improving End-to-End Neural Diarization Using Conversational Summary Representations标题：使用会话式摘要表示改进端到端神经二元化链接：https://arxiv.org/abs/2306.13863作者：Samuel J. Broughton,Lahiru Samarakoon机构：Fano Labs, Hong Kong SAR, China备注：5 pages, 1 figure, INTERSPEECH 2023摘要：说话人日志化是一项涉及按说话人身份划分音频记录的任务。端到端神经日志化与基于编码器-解码器的吸引子计算（EEND-EDA）旨在通过直接输出灵活数量的扬声器的日志化结果来解决这个问题。目前，负责生成说话者吸引子的EDA模块以零向量为条件，不向网络提供相关信息。在这项工作中，我们扩展EEND-EDA通过替换输入零向量的解码器学习会话摘要表示。更新的EDA模块基于话语级信息顺序地生成说话者吸引子。我们提出了三种方法来初始化的摘要向量，并进行调查不同的输入记录长度。在一系列公开的测试集上，与基线相比，我们的模型实现了1.90%的绝对DER性能改进。摘要：Speaker diarization is a task concerned with partitioning an audio recording by speaker identity. End-to-end neural diarization with encoder-decoder based attractor calculation (EEND-EDA) aims to solve this problem by directly outputting diarization results for a flexible number of speakers. Currently, the EDA module responsible for generating speaker-wise attractors is conditioned on zero vectors providing no relevant information to the network. In this work, we extend EEND-EDA by replacing the input zero vectors to the decoder with learned conversational summary representations. The updated EDA module sequentially generates speaker-wise attractors based on utterance-level information. We propose three methods to initialize the summary vector and conduct an investigation into varying input recording lengths. On a range of publicly available test sets, our model achieves an absolute DER performance improvement of 1.90 % when compared to the baseline.【11】 Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers标题：基于多模式双注意力转换器的跨语言语音情感识别链接：https://arxiv.org/abs/2306.13804作者：Syed Aun Muhammad Zaidi,Siddique Latif,Junaid Qadi机构：Qatar University备注：Under Review IEEE TMM摘要：尽管语音情感识别（SER）的最新进展，国家的最先进的系统是无法实现改进的性能，在跨语言设置。在本文中，我们提出了一个多模态双注意力Transformer（MDAT）模型，以提高跨语言SER。我们的模型利用预训练模型进行多模态特征提取，并配备了双重注意机制，包括图形注意和共同注意，以捕获不同模态之间的复杂依赖关系，并使用最少的目标语言数据实现改进的跨语言SER结果。此外，我们的模型还利用Transformer编码器层的高层次的特征表示，以提高情感分类的准确性。以这种方式，MDAT在各个阶段执行特征表示的细化，并向分类层提供情感显著特征。这种新的方法还确保了特定于情态的情感信息的保存，同时增强了跨情态和跨语言的交互。我们评估我们的模型的性能四个公开的SER数据集，并建立其优越的有效性相比，最近的方法和基线模型。摘要：Despite the recent progress in speech emotion recognition (SER), state-of-the-art systems are unable to achieve improved performance in cross-language settings. In this paper, we propose a Multimodal Dual Attention Transformer (MDAT) model to improve cross-language SER. Our model utilises pre-trained models for multimodal feature extraction and is equipped with a dual attention mechanism including graph attention and co-attention to capture complex dependencies across different modalities and achieve improved cross-language SER results using minimal target language data. In addition, our model also exploits a transformer encoder layer for high-level feature representation to improve emotion classification accuracy. In this way, MDAT performs refinement of feature representation at various stages and provides emotional salient features to the classification layer. This novel approach also ensures the preservation of modality-specific emotional information while enhancing cross-modality and cross-language interactions. We assess our model's performance on four publicly available SER datasets and establish its superior effectiveness compared to recent approaches and baseline models.【12】 Modulation Graphs in Popular Music标题：流行音乐中的调制图链接：https://arxiv.org/abs/2306.13691作者：Jason I. Brown,Ian George机构：Department of Mathematics and Statistics, Dalhousie University备注：17 pages, 6 figures, 6 tables摘要：本文运用图论的方法，从理论和应用两个方面探讨了音乐的调性调制概念。我们定义（枢轴）调制图的基础上，在流行音乐中使用的常见尺度。这些图的性质和参数进行了讨论。我们还调查了披头士作品中列侬-麦卡特尼歌曲的正典调制图。我们的方法可以提供作曲家与数学见解的枢轴调制。摘要：In this paper, graph theory is used to explore the musical notion of tonal modulation, in theory and application. We define (pivot) modulation graphs based on the common scales used in popular music. Properties and parameters of these graphs are discussed. We also investigate modulation graphs for the canon of Lennon-McCartney songs in the works of The Beatles. Our approach may provide composers with mathematical insights into pivot modulation.【13】 The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios标题：CHINE-7 DASR挑战：多种设备在不同场景下的远程会议转录链接：https://arxiv.org/abs/2306.13734作者：Samuele Cornell,Matthew Wiesner,Shinji Watanabe,Desh Raj,Xuankai Chang,Paola Garcia,Yoshiki Masuyama,Zhong-Qiu Wang,Stefano Squartini,Sanjeev Khudanpur机构：Italy ,Carnegie Mellon University, USA, Johns Hopkins University, USA ,Tokyo Metropolitan University, Japan摘要：CHiME挑战在鲁棒语音识别（ASR）系统的开发和评估中发挥了重要作用。我们在第7次CHiME挑战中引入CHiME-7远程ASR（DASR）任务。该任务包括在具有多个并且可能是异构的记录设备的远场设置中的联合ASR和diarization。与之前的挑战不同，我们在3种不同的场景下评估系统：CHiME-6、DiPCo和混合器6。目标是让参与者设计一个单一的系统，可以在没有先验信息的情况下推广到不同的阵列几何形状和用例。与早期CHiME迭代的另一个不同之处在于，参与者可以使用开源的预训练模型和数据集。在本文中，我们详细描述了挑战的设计，动机和基本的研究问题。我们还提出了基线系统，这是完全阵列拓扑不可知的，具有多通道日记，通道选择，引导源分离和一个强大的ASR模型，利用自监督语音表示（SSLR）。摘要：The CHiME challenges have played a significant role in the development and evaluation of robust speech recognition (ASR) systems. We introduce the CHiME-7 distant ASR (DASR) task, within the 7th CHiME challenge. This task comprises joint ASR and diarization in far-field settings with multiple, and possibly heterogeneous, recording devices. Different from previous challenges, we evaluate systems on 3 diverse scenarios: CHiME-6, DiPCo, and Mixer 6. The goal is for participants to devise a single system that can generalize across different array geometries and use cases with no a-priori information. Another departure from earlier CHiME iterations is that participants are allowed to use open-source pre-trained models and datasets. In this paper, we describe the challenge design, motivation, and fundamental research questions in detail. We also present the baseline system, which is fully array-topology agnostic and features multi-channel diarization, channel selection, guided source separation and a robust ASR model that leverages self-supervised speech representations (SSLR).【14】 Large-scale unsupervised audio pre-training for video-to-speech synthesis标题：视频到语音合成的大规模无监督音频预训练链接：https://arxiv.org/abs/2306.15464
作者：Triantafyllos Kefalas,Yannis Panagakis,Maja Pantic机构：uk)Triantafyllos Kefalas and Maja Pantic are with the Department of Comput-ing, UKYannis Panagakis is with the Department of Informatics and Telecommu-nications, University of Athens备注：Submitted to IEEE摘要：视频到语音合成是从说话者的无声视频重构语音信号的任务。迄今为止，大多数已建立的方法涉及两步过程，其中首先提取来自视频的中间表示（诸如频谱图），然后将其传递到声码器以产生原始音频。一些最近的工作集中在端到端合成上，由此原始音频和任何中间表示的生成被联合执行。所有这些方法都涉及对几乎完全来自视听数据集的数据进行训练，即每个音频样本具有相应的视频样本。这排除了使用可能不具有对应的视觉模态（例如，视觉模态）的丰富的仅音频数据集。有声读物、无线电播客、语音识别数据集等），以及音频机器学习社区多年来开发的纯音频架构。在本文中，我们建议在24 kHz的超过3,500小时的音频数据上训练编码器-解码器模型，然后使用预训练的解码器来初始化音频解码器，以进行视频到语音合成任务。预训练步骤仅使用音频样本，并且不需要来自其他模态（视觉、文本）的标签或对应样本。我们证明，这个预训练步骤提高了重建的语音，这是一个未探索的方式来提高质量的发生器在一个跨模态的任务，而只需要从一个模态的样本。我们进行实验，使用原始音频和梅尔频谱图作为目标输出和基准我们的模型与现有的工作。摘要：Video-to-speech synthesis is the task of reconstructing the speech signal from a silent video of a speaker. Most established approaches to date involve a two-step process, whereby an intermediate representation from the video, such as a spectrogram, is extracted first and then passed to a vocoder to produce the raw audio. Some recent work has focused on end-to-end synthesis, whereby the generation of raw audio and any intermediate representations is performed jointly. All such approaches involve training on data from almost exclusively audio-visual datasets, i.e. every audio sample has a corresponding video sample. This precludes the use of abundant audio-only datasets which may not have a corresponding visual modality (e.g. audiobooks, radio podcasts, speech recognition datasets etc.), as well as audio-only architectures that have been developed by the audio machine learning community over the years. In this paper we propose to train encoder-decoder models on more than 3,500 hours of audio data at 24kHz, and then use the pre-trained decoders to initialize the audio decoders for the video-to-speech synthesis task. The pre-training step uses audio samples only and does not require labels or corresponding samples from other modalities (visual, text). We demonstrate that this pre-training step improves the reconstructed speech and that it is an unexplored way to improve the quality of the generator in a cross-modal task while only requiring samples from one of the modalities. We conduct experiments using both raw audio and mel spectrograms as target outputs and benchmark our models with existing work.【15】 RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Music标题：RMVPE：一种稳健的复调音乐基音估计模型链接：https://arxiv.org/abs/2306.15412
作者：Haojie Wei,Xueke Cao,Tangpeng Dan,Yueguo Chen机构：School of Information, Renmin University of China, Beijing, China, Institute of Information Science, Beijing Jiaotong University, Beijing, China备注：This paper has been accepted by INTERSPEECH 2023摘要：基音是音乐音频处理中的一个重要的高级特征。然而，在复调音乐中，由于伴奏的存在，提取音高更具挑战性。为了消除伴奏的影响，大多数以前的方法采用音乐源分离模型，以获得干净的人声从复调音乐之前预测音高。因此，基音估计的性能受到音乐源分离模型的影响。为了解决这个问题，并直接从复调音乐中提取音高，我们提出了一个强大的模型RMVPE。该模型能有效地提取复调音乐中的隐含特征，准确地预测出复调音乐中的音高。实验结果表明，RMVPE的原始音高精度（RPA）和原始色度精度（RCA）方面的优越性。此外，用不同类型的噪声进行的实验表明，RMVPE在所有信噪比（SNR）水平上都是鲁棒的。RMVPE的代码可在此URL中获得。https: github.com Dream-High RMVPE.摘要：Vocal pitch is an important high-level feature in music audio processing. However, extracting vocal pitch in polyphonic music is more challenging due to the presence of accompaniment. To eliminate the influence of the accompaniment, most previous methods adopt music source separation models to obtain clean vocals from polyphonic music before predicting vocal pitches. As a result, the performance of vocal pitch estimation is affected by the music source separation models. To address this issue and directly extract vocal pitches from polyphonic music, we propose a robust model named RMVPE. This model can extract effective hidden features and accurately predict vocal pitches from polyphonic music. The experimental results demonstrate the superiority of RMVPE in terms of raw pitch accuracy (RPA) and raw chroma accuracy (RCA). Additionally, experiments conducted with different types of noise show that RMVPE is robust across all signal-to-noise ratio (SNR) levels. The code of RMVPE is available at this URL. https: github.com Dream-High RMVPE.【16】 Multi-perspective Information Fusion Res2Net with RandomSpecmix for Fake Speech Detection标题：基于RandomspecMix的多视角信息融合Res2Net假语音检测链接：https://arxiv.org/abs/2306.15389
作者：Shunbo Dong,Jun Xue,Cunhang Fan,Kang Zhu,Yujie Chen,Zhao Lv机构：Anhui Province Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University (AHU), Jiulong Road, Hefei, China备注：Accepted by DADA2023摘要：在本文中，我们提出了多视角信息融合（MPIF）Res 2Net与随机频谱混合的假语音检测（FSD）。该系统的主要目的是提高模型的能力，学习准确的伪造信息，FSD任务在低质量的情况下。随机Spectmix是一种数据增强方法，其任务是提高模型的泛化能力，增强模型定位判别信息的能力。Spectmix在不引入其他数据的情况下，将频谱图的频率维度信息剪切粘贴在同一批样本中，有助于模型定位真正有用的信息。同时，我们随机选择样本进行扩增，以减少数据扩增直接改变所有数据的影响。一旦达到帮助模型定位信息的目的，减少不必要的信息也很重要。MPIF-Res 2Net的作用是减少冗余干扰信息。从单一角度看的欺骗信息总是相似的，因此学习这种相似信息的模型会产生冗余的欺骗线索，干扰真正的区分信息。提出的MPIF-Res 2Net融合了不同角度的信息，使模型学习的信息更加多样，从而减少了相似信息造成的冗余，避免了对区分信息学习的干扰。ASVspoof 2021 LA数据集的结果证明了我们提出的方法的有效性，EER和min-tDCF分别为3.29%和0.2557。摘要：In this paper, we propose the multi-perspective information fusion (MPIF) Res2Net with random Specmix for fake speech detection (FSD). The main purpose of this system is to improve the model's ability to learn precise forgery information for FSD task in low-quality scenarios. The task of random Specmix, a data augmentation, is to improve the generalization ability of the model and enhance the model's ability to locate discriminative information. Specmix cuts and pastes the frequency dimension information of the spectrogram in the same batch of samples without introducing other data, which helps the model to locate the really useful information. At the same time, we randomly select samples for augmentation to reduce the impact of data augmentation directly changing all the data. Once the purpose of helping the model to locate information is achieved, it is also important to reduce unnecessary information. The role of MPIF-Res2Net is to reduce redundant interference information. Deceptive information from a single perspective is always similar, so the model learning this similar information will produce redundant spoofing clues and interfere with truly discriminative information. The proposed MPIF-Res2Net fuses information from different perspectives, making the information learned by the model more diverse, thereby reducing the redundancy caused by similar information and avoiding interference with the learning of discriminative information. The results on the ASVspoof 2021 LA dataset demonstrate the effectiveness of our proposed method, achieving EER and min-tDCF of 3.29% and 0.2557, respectively.【17】 3D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglement标题：3D-Speaker：一个大规模的多设备、多距离、多方言的语音表示解缠语料库链接：https://arxiv.org/abs/2306.15354
作者：Siqi Zheng,Luyao Cheng,Yafeng Chen,Hui Wang,Qian Chen机构：DAMO Academy, Alibaba Group摘要：从语音话语中提取不相关信息是语音学界的一个重要研究课题。不同的语音相关的任务集中在提取不同的语音表示，同时尽量减少其他不相关的信息的影响。我们提出了一个大规模的语音语料库，以促进语音表征去纠缠的研究。3D-Speaker包含超过10，000个扬声器，每个扬声器由位于不同距离的多个设备同时录制，并且一些扬声器正在讲多种方言。多维音频数据的受控组合产生语音表示纠缠的多样化混合的矩阵，从而激发有趣的方法来解开它们。3D-Speaker的多域性质也使其成为评估大型通用语音模型和域外学习和自监督学习的实验方法的合适资源。https: 3dspeaker.github.io摘要：Disentangling uncorrelated information in speech utterances is a crucial research topic within speech community. Different speech-related tasks focus on extracting distinct speech representations while minimizing the affects of other uncorrelated information. We present a large-scale speech corpus to facilitate the research of speech representation disentanglement. 3D-Speaker contains over 10,000 speakers, each of whom are simultaneously recorded by multiple Devices, locating at different Distances, and some speakers are speaking multiple Dialects. The controlled combinations of multi-dimensional audio data yield a matrix of a diverse blend of speech representation entanglement, thereby motivating intriguing methods to untangle them. The multi-domain nature of 3D-Speaker also makes it a suitable resource to evaluate large universal speech models and experiment methods of out-of-domain learning and self-supervised learning. https: 3dspeaker.github.io【18】 TranssionADD: A multi-frame reinforcement based sequence tagging model for audio deepfake detection标题：TranssionADD：一种基于多帧强化的音频深度伪检测序列标注模型链接：https://arxiv.org/abs/2306.15212
作者：Jie Liu,Zhiba Su,Hui Huang,Caiyan Wan,Quanxiu Wang,Jiangli Hong,Benlai Tang,Fengjie Zhu机构：Department of AI Technology, Transsion摘要：由于端到端语音建模技术的最新进展，模仿和克隆用户的声音变得越来越可行。这导致区分真实音频段和伪造音频段的重大挑战。为了解决用户语音滥用和误用的问题，第二届音频Deepfake检测挑战赛（ADD 2023）旨在检测和分析deepfake语音。具体地，被称为操纵区域位置（RL）的轨道2旨在精确定位音频中的操纵区域的位置，其可以存在于真实的和生成的音频片段两者中。我们提出了我们的新的TranssionADD系统作为一个具有挑战性的问题，模型的鲁棒性和音频段离群值的跟踪竞争的解决方案。我们的系统提供了三个独特的贡献：1）我们将序列标记任务适应于音频深度伪造检测; 2）通过各种数据增强技术提高模型的泛化能力; 3）我们结合多帧检测（MFD）模块来克服由单个帧提供的有限表示，并使用隔离帧惩罚（IFP）损失来处理片段中的离群值。我们最好的提交在Track 2中获得了第二名，证明了我们提出的系统的有效性和鲁棒性。摘要：Thanks to recent advancements in end-to-end speech modeling technology, it has become increasingly feasible to imitate and clone a user s voice. This leads to a significant challenge in differentiating between authentic and fabricated audio segments. To address the issue of user voice abuse and misuse, the second Audio Deepfake Detection Challenge (ADD 2023) aims to detect and analyze deepfake speech utterances. Specifically, Track 2, named the Manipulation Region Location (RL), aims to pinpoint the location of manipulated regions in audio, which can be present in both real and generated audio segments. We propose our novel TranssionADD system as a solution to the challenging problem of model robustness and audio segment outliers in the trace competition. Our system provides three unique contributions: 1) we adapt sequence tagging task for audio deepfake detection; 2) we improve model generalization by various data augmentation techniques; 3) we incorporate multi-frame detection (MFD) module to overcome limited representation provided by a single frame and use isolated-frame penalty (IFP) loss to handle outliers in segments. Our best submission achieved 2nd place in Track 2, demonstrating the effectiveness and robustness of our proposed system.【19】 A Deep Learning Architecture with Spatio-Temporal Focusing for Detecting Respiratory Anomalies标题：一种具有时空聚焦的呼吸异常检测深度学习结构链接：https://arxiv.org/abs/2306.14929
作者：Dat Ngo,Lam Pham,Huy Phan,Minh Tran,Delaram Jarchi机构：. School of Computer Science and Electronic Engineering, University of Essex, UK.备注：arXiv admin note: text overlap with arXiv:2303.04104摘要：本文提出了一种应用于从呼吸声记录中检测异常的深度学习系统。我们的系统首先使用连续小波变换进行音频特征提取。该变换将呼吸声输入转换成二维频谱图，其中呈现频谱和时间特征两者。然后，我们提出的深度学习架构受到基于Inception-residual的主干的启发，执行时空聚焦和多头注意机制来对呼吸异常进行分类。在这项工作中，我们评估了我们提出的模型的基准SPRSound（开源上海交通大学儿科呼吸声）数据库提出的IEEE BioCAS 2023挑战。关于通过平均分数和谐波分数之间的平均值计算的分数，我们的鲁棒系统在任务1-1、1-2、2-1和2-2中分别以0.810、0.667、0.744和0.608的分数实现了Top-1性能。摘要：This paper presents a deep learning system applied for detecting anomalies from respiratory sound recordings. Our system initially performs audio feature extraction using Continuous Wavelet transformation. This transformation converts the respiratory sound input into a two-dimensional spectrogram where both spectral and temporal features are presented. Then, our proposed deep learning architecture inspired by the Inception-residual-based backbone performs the spatial-temporal focusing and multi-head attention mechanism to classify respiratory anomalies. In this work, we evaluate our proposed models on the benchmark SPRSound (The Open-Source SJTU Paediatric Respiratory Sound) database proposed by the IEEE BioCAS 2023 challenge. As regards the Score computed by an average between the average score and harmonic score, our robust system has achieved Top-1 performance with Scores of 0.810, 0.667, 0.744, and 0.608 in Tasks 1-1, 1-2, 2-1, and 2-2, respectively.【20】 Post-Processing Independent Evaluation of Sound Event Detection Systems标题：声事件检测系统的后处理独立评估链接：https://arxiv.org/abs/2306.15440
作者：Janek Ebbers,Reinhold Haeb-Umbach,Romain Serizel机构：Paderborn University, Department of Communications Engineering, Paderborn, Germany, Universit´e de Lorraine, CNRS, Inria, Loria, F-, Nancy, France备注：submitted to DCASE Workshop 2023摘要：由于声音事件检测（SED）系统的应用要求的高度变化，仅在单一操作模式下评估系统是不够的。因此，社区最近采用了复调声音检测分数（PSDS）作为评估指标，这是PSD接收器工作特性（PSD-ROC）下的归一化面积。它总结了由改变用于将系统输出分数转换为二进制检测输出的决策阈值而产生的一系列操作模式上的系统性能。因此，它提供了整体系统行为的更完整的画面，并且较少受到特定阈值调整的影响。然而，除了决策阈值之外，还存在可以改变以进入另一操作模式的后处理。在本文中，我们提出了后处理独立的PSDS（piPSDS）的PSDS的泛化。这里，后处理独立的PSD-ROC包括来自具有变化的决策阈值的变化的后处理的操作点。因此，其总结了SED系统的甚至更多操作模式，并且允许系统比较，而不需要实施后处理并且没有由于不同后处理而导致的偏差。虽然piPSDS原则上可以结合不同类型的后处理，但作为第一步，我们听到了今年DCASE挑战任务4a系统的中值滤波器独立PSDS（miPSDS）结果。源代码在我们的sed_scores_eval包（https：github.com fgnt sed_scores_eval）中公开提供。摘要：Due to the high variation in the application requirements of sound event detection (SED) systems, it is not sufficient to evaluate systems only in a single operating mode. Therefore, the community recently adopted the polyphonic sound detection score (PSDS) as an evaluation metric, which is the normalized area under the PSD receiver operating characteristic (PSD-ROC). It summarizes the system performance over a range of operating modes resulting from varying the decision threshold that is used to translate the system output scores into a binary detection output. Hence, it provides a more complete picture of the overall system behavior and is less biased by specific threshold tuning. However, besides the decision threshold there is also the post-processing that can be changed to enter another operating mode. In this paper we propose the post-processing independent PSDS (piPSDS) as a generalization of the PSDS. Here, the post-processing independent PSD-ROC includes operating points from varying post-processings with varying decision thresholds. Thus, it summarizes even more operating modes of an SED system and allows for system comparison without the need of implementing a post-processing and without a bias due to different post-processings. While piPSDS can in principle combine different types of post-processing, we hear, as a first step, present median filter independent PSDS (miPSDS) results for this year's DCASE Challenge Task4a systems. Source code is publicly available in our sed_scores_eval package (https: github.com fgnt sed_scores_eval).【21】 GenerTTS: Pronunciation Disentanglement for Timbre and Style Generalization in Cross-Lingual Text-to-Speech标题：GenerTTS：跨语言文语转换中音色和风格泛化的发音解缠链接：https://arxiv.org/abs/2306.15304
作者：Yahuan Cong,Haoyu Zhang,Haopeng Lin,Shichao Liu,Chunfeng Wang,Yi Ren,Xiang Yin,Zejun Ma机构：ByteDance备注：Accepted by INTERSPEECH 2023摘要：跨语言音色和风格可泛化的文本到语音（TTS）旨在合成具有特定参考音色或风格的语音，该参考音色或风格从未在目标语言中训练过。它面临以下挑战：1）音色和发音是相关的，因为特定说话者的多语言语音通常难以获得;（2）语体和语音是混合的，因为言语语体既有语言不可知的部分，也有语言特有的部分。为了应对这些挑战，我们提出了GenerTTS，主要包括以下工作：1）我们精心设计了一个基于HuBERT的信息瓶颈，以解开音色和发音 风格的纠缠; 2）通过最小化风格与语言之间的互信息，去除风格嵌入中的语言相关信息。实验表明，GenerTTS优于基线系统的风格相似性和发音准确性，并实现跨语言的音色和风格的泛化。摘要：Cross-lingual timbre and style generalizable text-to-speech (TTS) aims to synthesize speech with a specific reference timbre or style that is never trained in the target language. It encounters the following challenges: 1) timbre and pronunciation are correlated since multilingual speech of a specific speaker is usually hard to obtain; 2) style and pronunciation are mixed because the speech style contains language-agnostic and language-specific parts. To address these challenges, we propose GenerTTS, which mainly includes the following works: 1) we elaborately design a HuBERT-based information bottleneck to disentangle timbre and pronunciation style; 2) we minimize the mutual information between style and language to discard the language-specific information in the style embedding. The experiments indicate that GenerTTS outperforms baseline systems in terms of style similarity and pronunciation accuracy, and enables cross-lingual timbre and style generalization.【22】 Wespeaker baselines for VoxSRC2023标题：VoxSRC2023的Wespeaker基准链接：https://arxiv.org/abs/2306.15161
作者：Shuai Wang,Chengdong Liang,Xu Xiang,Bing Han,Zhengyang Chen,Hongji Wang,Wen Ding机构：Wespeaker Team, WeNet Open Source Community摘要：本报告展示了使用wespeaker工具包进行VoxSRC2023挑战所取得的成果。我们的目标是为参与者，特别是那些经验有限的人，提供清晰而直接的指导方针来开发他们的初始系统。通过结构良好的食谱和强大的结果，我们希望为所有感兴趣的人提供一个方便和足够好的起点。在本报告中，我们描述了使用预训练模型在VoxSRC2023开发集上获得的结果，您可以检查CodaLab评估服务器以获得评估集上的结果。摘要：This report showcases the results achieved using the wespeaker toolkit for the VoxSRC2023 Challenge. Our aim is to provide participants, especially those with limited experience, with clear and straightforward guidelines to develop their initial systems. Via well-structured recipes and strong results, we hope to offer an accessible and good enough start point for all interested individuals. In this report, we describe the results achieved on the VoxSRC2023 dev set using the pretrained models, you can check the CodaLab evaluation server for the results on the evaluation set.【1】 Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems标题：一致性语音识别系统的因子化说话人-环境自适应训练链接：https://arxiv.org/abs/2306.14608作者：Jiajun Deng,Guinan Li,Xurong Xie,Zengrui Jin,Mingyu Cui,Tianzi Wang,Shujie Hu,Mengzhe Geng,Xunying Liu机构：The Chinese University of Hong Kong, Hong Kong SAR, China, Institute of Software, Chinese Academy of Sciences, Beijing, China备注：Accepted by INTERSPEECH 2023摘要：自然语音中丰富的可变性来源对当前数据密集型语音识别技术提出了重大挑战。为了对说话人和环境水平的多样性进行建模，本文提出了一种新的贝叶斯分解的说话人-环境自适应训练和测试时间自适应方法，用于Conformer ASR模型。扬声器和环境级特性分别建模使用紧凑的隐藏输出变换，然后线性或分层组合，以表示任何扬声器-环境组合。贝叶斯学习被进一步用于对自适应参数不确定性进行建模。300小时WHAM噪声损坏交换机数据的实验表明，因子化自适应始终优于基线和扬声器标签只适应构象高达3.1%的绝对（10.4%相对）字错误率降低。进一步的分析表明，所提出的方法提供了潜在的快速适应看不见的扬声器环境条件。摘要：Rich sources of variability in natural speech present significant challenges to current data intensive speech recognition technologies. To model both speaker and environment level diversity, this paper proposes a novel Bayesian factorised speaker-environment adaptive training and test time adaptation approach for Conformer ASR models. Speaker and environment level characteristics are separately modeled using compact hidden output transforms, which are then linearly or hierarchically combined to represent any speaker-environment combination. Bayesian learning is further utilized to model the adaptation parameter uncertainty. Experiments on the 300-hr WHAM noise corrupted Switchboard data suggest that factorised adaptation consistently outperforms the baseline and speaker label only adapted Conformers by up to 3.1% absolute (10.4% relative) word error rate reductions. Further analysis shows the proposed method offers potential for rapid adaption to unseen speaker-environment conditions.【2】 Community Detection Graph Convolutional Network for Overlap-Aware Speaker Diarization标题：基于重叠感知说话人二值化的社区检测图卷积网络链接：https://arxiv.org/abs/2306.14530作者：Jie Wang,Zhicong Chen,Haodong Zhou,Lin Li,Qingyang Hong机构：School of Electronic Science and Engineering, Xiamen University, China, School of Informatics, Xiamen University, China备注：None摘要：聚类算法在说话人拨号系统中起着至关重要的作用。然而，传统的聚类算法遭受说话人嵌入的复杂分布和缺乏挖掘会话中说话人之间潜在的关系。我们提出了一种新的基于图的聚类方法称为社区检测图卷积网络（CDGCN），以提高说话人拨号系统的性能。基于CDGCN的聚类方法包括图生成、子图检测和基于图的重叠语音检测（Graph-OSD）。首先，图生成细化语音片段之间的局部链接。其次，子图检测找到说话人图的最优全局划分。最后，我们认为说话人聚类重叠感知的说话人diarization作为一个重叠社区检测任务和设计了图形OSD组件输出重叠感知标签。通过捕获局部和全局信息，CDGCN聚类的说话人拨号系统优于传统的基于聚类的说话人拨号（CSD）系统DIHARD III语料库。摘要：The clustering algorithm plays a crucial role in speaker diarization systems. However, traditional clustering algorithms suffer from the complex distribution of speaker embeddings and lack of digging potential relationships between speakers in a session. We propose a novel graph-based clustering approach called Community Detection Graph Convolutional Network (CDGCN) to improve the performance of the speaker diarization system. The CDGCN-based clustering method consists of graph generation, sub-graph detection, and Graph-based Overlapped Speech Detection (Graph-OSD). Firstly, the graph generation refines the local linkages among speech segments. Secondly the sub-graph detection finds the optimal global partition of the speaker graph. Finally, we view speaker clustering for overlap-aware speaker diarization as an overlapped community detection task and design a Graph-OSD component to output overlap-aware labels. By capturing local and global information, the speaker diarization system with CDGCN clustering outperforms the traditional Clustering-based Speaker Diarization (CSD) systems on the DIHARD III corpus.【3】 The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios标题：CHINE-7 DASR挑战：多种设备在不同场景下的远程会议转录链接：https://arxiv.org/abs/2306.13734作者：Samuele Cornell,Matthew Wiesner,Shinji Watanabe,Desh Raj,Xuankai Chang,Paola Garcia,Yoshiki Masuyama,Zhong-Qiu Wang,Stefano Squartini,Sanjeev Khudanpur机构：Italy ,Carnegie Mellon University, USA, Johns Hopkins University, USA ,Tokyo Metropolitan University, Japan摘要：CHiME挑战在鲁棒语音识别（ASR）系统的开发和评估中发挥了重要作用。我们在第7次CHiME挑战中引入CHiME-7远程ASR（DASR）任务。该任务包括在具有多个并且可能是异构的记录设备的远场设置中的联合ASR和diarization。与之前的挑战不同，我们在3种不同的场景下评估系统：CHiME-6、DiPCo和混合器6。目标是让参与者设计一个单一的系统，可以在没有先验信息的情况下推广到不同的阵列几何形状和用例。与早期CHiME迭代的另一个不同之处在于，参与者可以使用开源的预训练模型和数据集。在本文中，我们详细描述了挑战的设计，动机和基本的研究问题。我们还提出了基线系统，这是完全阵列拓扑不可知的，具有多通道日记，通道选择，引导源分离和一个强大的ASR模型，利用自监督语音表示（SSLR）。摘要：The CHiME challenges have played a significant role in the development and evaluation of robust speech recognition (ASR) systems. We introduce the CHiME-7 distant ASR (DASR) task, within the 7th CHiME challenge. This task comprises joint ASR and diarization in far-field settings with multiple, and possibly heterogeneous, recording devices. Different from previous challenges, we evaluate systems on 3 diverse scenarios: CHiME-6, DiPCo, and Mixer 6. The goal is for participants to devise a single system that can generalize across different array geometries and use cases with no a-priori information. Another departure from earlier CHiME iterations is that participants are allowed to use open-source pre-trained models and datasets. In this paper, we describe the challenge design, motivation, and fundamental research questions in detail. We also present the baseline system, which is fully array-topology agnostic and features multi-channel diarization, channel selection, guided source separation and a robust ASR model that leverages self-supervised speech representations (SSLR).【4】 Mono-to-stereo through parametric stereo generation标题：通过参数立体声生成实现单声道到立体声链接：https://arxiv.org/abs/2306.14647作者：Joan Serrà,Davide Scaini,Santiago Pascual,Daniel Arteaga,Jordi Pons,Jeroen Breebaart,Giulio Cengarle机构：Dolby Laboratories备注：7 pages, 1 figure; accepted for ISMIR23摘要：从单声道音频信号生成立体声呈现是一项具有挑战性的开放式任务，特别是如果目标是获得具有声音元素的特定平移的逼真空间成像。在这项工作中，我们建议通过使用最近邻和深度网络方法预测参数立体声（PS）参数来将单声道转换为立体声。结合PS，我们还提出建模的任务生成方法，允许合成多个和同样合理的立体声再现从相同的单声道信号。为了实现这一目标，我们考虑了自回归和屏蔽令牌建模方法。我们提供的证据表明，拟议的基于PS的模型优于竞争力的经典去相关基线，在PS预测框架内，现代生成模型胜过同等的非生成模型。总的来说，我们的工作定位PS和生成建模作为单声道到立体声上混的强大和有吸引力的方法。还提供了这些方法的局限性的讨论。摘要：Generating a stereophonic presentation from a monophonic audio signal is a challenging open task, especially if the goal is to obtain a realistic spatial imaging with a specific panning of sound elements. In this work, we propose to convert mono to stereo by means of predicting parametric stereo (PS) parameters using both nearest neighbor and deep network approaches. In combination with PS, we also propose to model the task with generative approaches, allowing to synthesize multiple and equally-plausible stereo renditions from the same mono signal. To achieve this, we consider both autoregressive and masked token modelling approaches. We provide evidence that the proposed PS-based models outperform a competitive classical decorrelation baseline and that, within a PS prediction framework, modern generative models outshine equivalent non-generative counterparts. Overall, our work positions both PS and generative modelling as strong and appealing methodologies for mono-to-stereo upmixing. A discussion of the limitations of these approaches is also provided.【5】 Cross-Lingual Cross-Age Group Adaptation for Low-Resource Elderly Speech Emotion Recognition标题：低资源老年人语音情感识别的跨语言跨年龄组自适应链接：https://arxiv.org/abs/2306.14517作者：Samuel Cahyawijaya,Holy Lovenia,Willy Chung,Rita Frieske,Zihan Liu,Pascale Fung机构：The Hong Kong University of Science and Technology备注：Accepted in INTERSPEECH 2023摘要：语音情感识别在人机交互中起着至关重要的作用。然而，大多数语音情感识别研究都偏向于讲英语的成年人，这阻碍了其适用于不同语言和年龄段的其他人口统计群体。在这项工作中，我们分析了情绪识别在三种不同语言之间的可转移性-英语，汉语普通话和广东话;和两个不同的年龄组成年人和老年人。为了进行实验，我们开发了一个成年人和老年人的英语-普通话语音情感基准，BiMotion，和粤语语音情感数据集，YueMotion。本研究的结论是，不同的语言和年龄组需要特定的语音特征，从而使跨语言推理的方法是不合适的。然而，跨组数据增强仍然有利于正则化模型，语言距离对跨语言可移植性有重要影响。我们在www.example.com公开发布我们的代码https://github.com/HLTCHKUST/elderly_ser。摘要：Speech emotion recognition plays a crucial role in human-computer interactions. However, most speech emotion recognition research is biased toward English-speaking adults, which hinders its applicability to other demographic groups in different languages and age groups. In this work, we analyze the transferability of emotion recognition across three different languages--English, Mandarin Chinese, and Cantonese; and 2 different age groups--adults and the elderly. To conduct the experiment, we develop an English-Mandarin speech emotion benchmark for adults and the elderly, BiMotion, and a Cantonese speech emotion dataset, YueMotion. This study concludes that different language and age groups require specific speech features, thus making cross-lingual inference an unsuitable method. However, cross-group data augmentation is still beneficial to regularize the model, with linguistic distance being a significant influence on cross-lingual transferability. We release publicly release our code at https://github.com/HLTCHKUST/elderly_ser.【6】 The Singing Voice Conversion Challenge 2023标题：2023年歌声转换挑战赛链接：https://arxiv.org/abs/2306.14422作者：Wen-Chin Huang,Lester Phillip Violeta,Songxiang Liu,Jiatong Shi,Yusuke Yasuda,Tomoki Toda机构：Nagoya University, Japan, Tencent AI Lab, Carnegie Mellon University, USA摘要：我们介绍了语音转换挑战（VCC）系列的最新版本，这是一项每两年举办一次的科学活动，旨在比较和了解基于通用数据集的不同语音转换（VC）系统。今年，我们将重点转移到歌唱声音转换（SVC），因此将挑战命名为歌唱声音转换挑战（SVCC）。构建了一个新的数据库，用于两个任务，即域内和跨域SVC。挑战赛进行了两个月，我们总共收到了26份提交，包括2个基线。通过大规模的众包听力测试，我们观察到，对于这两个任务，虽然人类水平的自然度是由顶级系统实现的，但没有一个团队能够获得与目标说话者一样高的相似度分数。此外，如预期的，跨域SVC比域内SVC更难，特别是在相似性方面。我们还调查了现有的客观测量是否能够预测感知性能，并发现只有少数人可以达到显着的相关性。摘要：We present the latest iteration of the voice conversion challenge (VCC) series, a bi-annual scientific event aiming to compare and understand different voice conversion (VC) systems based on a common dataset. This year we shifted our focus to singing voice conversion (SVC), thus named the challenge the Singing Voice Conversion Challenge (SVCC). A new database was constructed for two tasks, namely in-domain and cross-domain SVC. The challenge was run for two months, and in total we received 26 submissions, including 2 baselines. Through a large-scale crowd-sourced listening test, we observed that for both tasks, although human-level naturalness was achieved by the top system, no team was able to obtain a similarity score as high as the target speakers. Also, as expected, cross-domain SVC is harder than in-domain SVC, especially in the similarity aspect. We also investigated whether existing objective measurements were able to predict perceptual performance, and found that only few of them could reach a significant correlation.【7】 Addressing Cold Start Problem for End-to-end Automatic Speech Scoring标题：解决端到端自动语音评分的冷启动问题链接：https://arxiv.org/abs/2306.14310作者：Jungbae Park,Seungtaek Choi机构：Riiid AI Research备注：Accepted at Interspeech 2023, 4 pages, 1 page for reference摘要：集成自动语音评分/评估系统已成为第二语言口语教育的一个重要方面。随着自我监督学习的进步，端到端语音评分方法已经显示出有希望的结果。然而，这项研究强调了在新的问题背景下，语音评分系统的性能显着下降，从而确定这是一个冷启动问题的项目。随着冷启动现象的发现，本文试图通过以下方法缓解该问题：1）提示嵌入，2）使用BERT或CLIP模型的问题上下文嵌入，以及3）预训练的声学模型的选择。实验是在托业口语测试数据集上进行的，这些数据集是由专业的托业口语评估员对英语作为第二语言（ESL）的学习者进行评估的。结果表明，所提出的框架不仅表现出在冷启动环境中的鲁棒性，但也优于已知内容的基线。摘要：Integrating automatic speech scoring/assessment systems has become a critical aspect of second-language speaking education. With self-supervised learning advancements, end-to-end speech scoring approaches have exhibited promising results. However, this study highlights the significant decrease in the performance of speech scoring systems in new question contexts, thereby identifying this as a cold start problem in terms of items. With the finding of cold-start phenomena, this paper seeks to alleviate the problem by following methods: 1) prompt embeddings, 2) question context embeddings using BERT or CLIP models, and 3) choice of the pretrained acoustic model. Experiments are conducted on TOEIC speaking test datasets collected from English-as-a-second-language (ESL) learners rated by professional TOEIC speaking evaluators. The results demonstrate that the proposed framework not only exhibits robustness in a cold-start environment but also outperforms the baselines for known content.【8】 Aeroacoustic Source Localization标题：气动声源定位链接：https://arxiv.org/abs/2306.14276作者：Weicheng Xue,Bing Yang,Shaohong Jia机构：• This work applied the beamforming algorithm and the DAMAS algo-, rithm to perform sound source localization for numerical simulations, as well as real-world experiments for a ,., MW wind turbine.摘要：解卷积DAMAS算法可以有效地消除通常使用的波束形成定位算法中的误解，允许更准确地计算源位置以及强度。在求解线性方程组时，DAMAS算法考虑了不同位置的相互影响，减少甚至消除了旁瓣，产生了更精确的结果。 本文首先介绍了DAMAS算法的原理。然后，它适用于波束形成算法和DAMAS算法来模拟定位的单频源从1.5兆瓦的风力涡轮机，一个复杂的线源与文本“UCAS”和线源下游的翼型后缘。最后，工作提出了实验定位结果的源的1.5兆瓦的风力涡轮机使用波束形成算法和DAMAS算法。摘要：The deconvolutional DAMAS algorithm can effectively eliminate the misconceptions in the usually-used beamforming localization algorithm, allowing for more accurate calculation of the source location as well as the intensity. When solving a linear system of equations, the DAMAS algorithm takes into account the mutual influence of different locations, reducing or even eliminating sidelobes and producing more accurate results. This work first introduces the principles of the DAMAS algorithm. Then it applies both the beamforming algorithm and the DAMAS algorithm to simulate the localization of a single-frequency source from a 1.5 MW wind turbine, a complex line source with the text "UCAS" and a line source downstream of an airfoil trailing edge. Finally, the work presents experimental localization results of the source of a 1.5 MW wind turbine using both the beamforming algorithm and the DAMAS algorithm.【9】 PrimaDNN': A Characteristics-aware DNN Customization for Singing Technique Detection标题：PrimaDNN‘：一种用于歌唱技巧检测的特征感知DNN定制链接：https://arxiv.org/abs/2306.14191作者：Yuya Yamamoto,Juhan Nam,Hiroko Terasawa机构：Doctoral Program in Informatics, University of Tsukuba, Ibaraki, Japan, Graduate School of Culture Technology, KAIST, Daejeon, South Korea备注：Accepted at EUSIPCO 2023摘要：专业歌手调整他们的声音音色或音高，使他们的声乐表演更具表现力。这种波动被称为歌唱技巧。从音轨中自动检测歌唱技巧可以有益于理解每个歌唱者如何表达表演，然而由于歌唱技巧的广泛多样性，这也可能是困难的。深度神经网络（DNN）模型可以处理这种变化;然而，考虑数据的特性可能会提高歌唱技术检测的性能。在本文中，我们提出了PrimaDNN，这是一种具有面向特征的改进的CRNN模型。该模型的特点是：1）基于辅助音高信息和多分辨率梅尔频谱图的输入特征表示，2）基于挤压和激励（SENet）和实例归一化的卷积模块。在J-POP歌唱技术检测结果中，PrimaDNN在整体macro-F测量中取得了44.9%的最佳结果，与传统作品相比。我们还发现，每个组件的贡献不同，这取决于歌唱技术的类型。摘要：Professional vocalists modulate their voice timbre or pitch to make their vocal performance more expressive. Such fluctuations are called singing techniques. Automatic detection of singing techniques from audio tracks can be beneficial to understand how each singer expresses the performance, yet it can also be difficult due to the wide variety of the singing techniques. A deep neural network (DNN) model can handle such variety; however, there might be a possibility that considering the characteristics of the data improves the performance of singing technique detection. In this paper, we propose PrimaDNN, a CRNN model with a characteristics-oriented improvement. The features of the model are: 1) input feature representation based on auxiliary pitch information and multi-resolution mel spectrograms, 2) Convolution module based on the Squeeze-and-excitation (SENet) and the Instance normalization. In the results of J-POP singing technique detection, PrimaDNN achieved the best results of 44.9% at the overall macro-F measure, compared to conventional works. We also found that the contribution of each component varies depending on the type of singing technique.【10】 AV-SepFormer: Cross-Attention SepFormer for Audio-Visual Target Speaker Extraction标题：用于视听目标说话人提取的交叉注意SepFormer链接：https://arxiv.org/abs/2306.14170作者：Jiuxin Lin,Xinyu Cai,Heinrich Dinkel,Jun Chen,Zhiyong Yan,Yongqing Wang,Junbo Zhang,Zhiyong Wu,Yujun Wang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, Xiaomi Inc., Beijing, China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：Accepted by ICASSP2023摘要：视觉信息可以作为一个有效的线索，目标说话人提取（TSE），是至关重要的，以提高提取性能。在本文中，我们提出了AV-SepFormer，一个基于SepFormer的注意力双尺度模型，利用交叉和自我注意力融合和模型特征的音频和视觉。AV-SepFormer将音频特征分割成多个块，相当于视觉特征的长度。然后利用自注意和交叉注意对多模态特征进行建模和融合。此外，我们使用了一种新的2D位置编码，它引入了块之间和块内的位置信息，并提供了显着的收益超过传统的位置编码。我们的模型有两个关键优势：将音频分块特征的时间粒度与视觉特征同步，缓解了音视频采样率不一致带来的危害;通过组合自注意和交叉注意，特征融合和语音提取过程在注意范例内被统一。实验结果表明，AV-SepFormer算法明显优于现有的其他方法。摘要：Visual information can serve as an effective cue for target speaker extraction (TSE) and is vital to improving extraction performance. In this paper, we propose AV-SepFormer, a SepFormer-based attention dual-scale model that utilizes cross- and self-attention to fuse and model features from audio and visual. AV-SepFormer splits the audio feature into a number of chunks, equivalent to the length of the visual feature. Then self- and cross-attention are employed to model and fuse the multi-modal features. Furthermore, we use a novel 2D positional encoding, that introduces the positional information between and within chunks and provides significant gains over the traditional positional encoding. Our model has two key advantages: the time granularity of audio chunked feature is synchronized to the visual feature, which alleviates the harm caused by the inconsistency of audio and video sampling rate; by combining self- and cross-attention, feature fusion and speech extraction processes are unified within an attention paradigm. The experimental results show that AV-SepFormer significantly outperforms other existing methods.【11】 DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech标题：DSE-TTS：用于跨语言文语转换的双说话人嵌入链接：https://arxiv.org/abs/2306.14145作者：Sen Liu,Yiwei Guo,Chenpeng Du,Xie Chen,Kai Yu机构：MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China备注：Accepted to Interspeech 2023摘要：虽然对于语内语音合成可以获得高保真语音，但是跨语言文本到语音（CTTS）仍然远远不能令人满意，因为难以准确地保留说话者音色（即，语音合成）。说话者相似性）并从他们的第一语言中消除口音（即，原生性）。在本文中，我们证明了矢量量化（VQ）的声学特征包含较少的说话人信息比梅尔频谱图。基于这一发现，我们提出了一种新的双说话人嵌入TTS（DSE-TTS）框架CTTS与真实的说话风格。这里，一个嵌入被馈送到声学模型以学习语言说话风格，而另一个被集成到声码器中以模仿目标说话者的音色。实验表明，通过结合这两种嵌入，DSE-TTS显着优于国家的最先进的SANE-TTS在跨语言合成，特别是在本土化方面。摘要：Although high-fidelity speech can be obtained for intralingual speech synthesis, cross-lingual text-to-speech (CTTS) is still far from satisfactory as it is difficult to accurately retain the speaker timbres(i.e. speaker similarity) and eliminate the accents from their first language(i.e. nativeness). In this paper, we demonstrated that vector-quantized(VQ) acoustic feature contains less speaker information than mel-spectrogram. Based on this finding, we propose a novel dual speaker embedding TTS (DSE-TTS) framework for CTTS with authentic speaking style. Here, one embedding is fed to the acoustic model to learn the linguistic speaking style, while the other one is integrated into the vocoder to mimic the target speaker's timbre. Experiments show that by combining both embeddings, DSE-TTS significantly outperforms the state-of-the-art SANE-TTS in cross-lingual synthesis, especially in terms of nativeness.【12】 An Analysis of Personalized Speech Recognition System Development for the Deaf and Hard-of-Hearing标题：聋哑人个性化语音识别系统开发分析链接：https://arxiv.org/abs/2306.13953作者：Lester Phillip Violeta,Tomoki Toda机构：Nagoya University, Japan备注：Submitted to APSIPA 2023摘要：耳聋或重听（DHH）说话者通常具有由耳聋引起的非典型言语。随着基于语音的设备和软件应用程序的支持不断增加，需要做更多的工作来使这些设备对每个人都具有包容性。要做到这一点，我们分析了开放可用的自动语音识别（ASR）工具与DHH日语说话者数据集的使用。由于这些开箱即用的ASR模型通常不表现良好的DHH语音，我们提供了创建个性化ASR系统的彻底分析。我们收集了一个大的DHH说话者数据集，四个说话者总计约28.05小时，并通过改变训练数据大小彻底分析了不同训练框架的性能。我们的研究结果表明，1000个话语（或1-2小时）从目标说话人已经可以显着提高模型的性能与最少的工作量所需的，因此我们建议研究人员收集至少1000个话语，使一个有效的个性化ASR系统。在1000个话语难以收集的情况下，我们还发现显着的改进，在使用先前提出的数据增强技术，如中间微调时，只有200个话语可用。摘要：Deaf or hard-of-hearing (DHH) speakers typically have atypical speech caused by deafness. With the growing support of speech-based devices and software applications, more work needs to be done to make these devices inclusive to everyone. To do so, we analyze the use of openly-available automatic speech recognition (ASR) tools with a DHH Japanese speaker dataset. As these out-of-the-box ASR models typically do not perform well on DHH speech, we provide a thorough analysis of creating personalized ASR systems. We collected a large DHH speaker dataset of four speakers totaling around 28.05 hours and thoroughly analyzed the performance of different training frameworks by varying the training data sizes. Our findings show that 1000 utterances (or 1-2 hours) from a target speaker can already significantly improve the model performance with minimal amount of work needed, thus we recommend researchers to collect at least 1000 utterances to make an efficient personalized ASR system. In cases where 1000 utterances is difficult to collect, we also discover significant improvements in using previously proposed data augmentation techniques such as intermediate fine-tuning when only 200 utterances are available.【13】 Improving End-to-End Neural Diarization Using Conversational Summary Representations标题：使用会话式摘要表示改进端到端神经二元化链接：https://arxiv.org/abs/2306.13863作者：Samuel J. Broughton,Lahiru Samarakoon机构：Fano Labs, Hong Kong SAR, China备注：5 pages, 1 figure, INTERSPEECH 2023摘要：说话人日志化是一项涉及按说话人身份划分音频记录的任务。端到端神经日志化与基于编码器-解码器的吸引子计算（EEND-EDA）旨在通过直接输出灵活数量的扬声器的日志化结果来解决这个问题。目前，负责生成说话者吸引子的EDA模块以零向量为条件，不向网络提供相关信息。在这项工作中，我们扩展EEND-EDA通过替换输入零向量的解码器学习会话摘要表示。更新的EDA模块基于话语级信息顺序地生成说话者吸引子。我们提出了三种方法来初始化的摘要向量，并进行调查不同的输入记录长度。在一系列公开的测试集上，与基线相比，我们的模型实现了1.90%的绝对DER性能改进。摘要：Speaker diarization is a task concerned with partitioning an audio recording by speaker identity. End-to-end neural diarization with encoder-decoder based attractor calculation (EEND-EDA) aims to solve this problem by directly outputting diarization results for a flexible number of speakers. Currently, the EDA module responsible for generating speaker-wise attractors is conditioned on zero vectors providing no relevant information to the network. In this work, we extend EEND-EDA by replacing the input zero vectors to the decoder with learned conversational summary representations. The updated EDA module sequentially generates speaker-wise attractors based on utterance-level information. We propose three methods to initialize the summary vector and conduct an investigation into varying input recording lengths. On a range of publicly available test sets, our model achieves an absolute DER performance improvement of 1.90 % when compared to the baseline.【14】 Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers标题：基于多模式双注意力转换器的跨语言语音情感识别链接：https://arxiv.org/abs/2306.13804作者：Syed Aun Muhammad Zaidi,Siddique Latif,Junaid Qadi机构：Qatar University备注：Under Review IEEE TMM摘要：尽管语音情感识别（SER）的最新进展，国家的最先进的系统是无法实现改进的性能，在跨语言设置。在本文中，我们提出了一个多模态双注意力Transformer（MDAT）模型，以提高跨语言SER。我们的模型利用预训练模型进行多模态特征提取，并配备了双重注意机制，包括图形注意和共同注意，以捕获不同模态之间的复杂依赖关系，并使用最少的目标语言数据实现改进的跨语言SER结果。此外，我们的模型还利用Transformer编码器层的高层次的特征表示，以提高情感分类的准确性。以这种方式，MDAT在各个阶段执行特征表示的细化，并向分类层提供情感显著特征。这种新的方法还确保了特定于情态的情感信息的保存，同时增强了跨情态和跨语言的交互。我们评估我们的模型的性能四个公开的SER数据集，并建立其优越的有效性相比，最近的方法和基线模型。摘要：Despite the recent progress in speech emotion recognition (SER), state-of-the-art systems are unable to achieve improved performance in cross-language settings. In this paper, we propose a Multimodal Dual Attention Transformer (MDAT) model to improve cross-language SER. Our model utilises pre-trained models for multimodal feature extraction and is equipped with a dual attention mechanism including graph attention and co-attention to capture complex dependencies across different modalities and achieve improved cross-language SER results using minimal target language data. In addition, our model also exploits a transformer encoder layer for high-level feature representation to improve emotion classification accuracy. In this way, MDAT performs refinement of feature representation at various stages and provides emotional salient features to the classification layer. This novel approach also ensures the preservation of modality-specific emotional information while enhancing cross-modality and cross-language interactions. We assess our model's performance on four publicly available SER datasets and establish its superior effectiveness compared to recent approaches and baseline models.【15】 Post-Processing Independent Evaluation of Sound Event Detection Systems标题：声事件检测系统的后处理独立评估链接：https://arxiv.org/abs/2306.15440
*与cs.SD语音【20】为同一篇
作者：Janek Ebbers,Reinhold Haeb-Umbach,Romain Serizel机构：Paderborn University, Department of Communications Engineering, Paderborn, Germany, Universit´e de Lorraine, CNRS, Inria, Loria, F-, Nancy, France备注：submitted to DCASE Workshop 2023摘要：由于声音事件检测（SED）系统的应用要求的高度变化，仅在单一操作模式下评估系统是不够的。因此，社区最近采用了复调声音检测分数（PSDS）作为评估指标，这是PSD接收器工作特性（PSD-ROC）下的归一化面积。它总结了由改变用于将系统输出分数转换为二进制检测输出的决策阈值而产生的一系列操作模式上的系统性能。因此，它提供了整体系统行为的更完整的画面，并且较少受到特定阈值调整的影响。然而，除了决策阈值之外，还存在可以改变以进入另一操作模式的后处理。在本文中，我们提出了后处理独立的PSDS（piPSDS）的PSDS的泛化。这里，后处理独立的PSD-ROC包括来自具有变化的决策阈值的变化的后处理的操作点。因此，其总结了SED系统的甚至更多操作模式，并且允许系统比较，而不需要实施后处理并且没有由于不同后处理而导致的偏差。虽然piPSDS原则上可以结合不同类型的后处理，但作为第一步，我们听到了今年DCASE挑战任务4a系统的中值滤波器独立PSDS（miPSDS）结果。源代码在我们的sed_scores_eval包（https：github.com fgnt sed_scores_eval）中公开提供。摘要：Due to the high variation in the application requirements of sound event detection (SED) systems, it is not sufficient to evaluate systems only in a single operating mode. Therefore, the community recently adopted the polyphonic sound detection score (PSDS) as an evaluation metric, which is the normalized area under the PSD receiver operating characteristic (PSD-ROC). It summarizes the system performance over a range of operating modes resulting from varying the decision threshold that is used to translate the system output scores into a binary detection output. Hence, it provides a more complete picture of the overall system behavior and is less biased by specific threshold tuning. However, besides the decision threshold there is also the post-processing that can be changed to enter another operating mode. In this paper we propose the post-processing independent PSDS (piPSDS) as a generalization of the PSDS. Here, the post-processing independent PSD-ROC includes operating points from varying post-processings with varying decision thresholds. Thus, it summarizes even more operating modes of an SED system and allows for system comparison without the need of implementing a post-processing and without a bias due to different post-processings. While piPSDS can in principle combine different types of post-processing, we hear, as a first step, present median filter independent PSDS (miPSDS) results for this year's DCASE Challenge Task4a systems. Source code is publicly available in our sed_scores_eval package (https: github.com fgnt sed_scores_eval).
【16】 GenerTTS: Pronunciation Disentanglement for Timbre and Style Generalization in Cross-Lingual Text-to-Speech标题：GenerTTS：跨语言文语转换中音色和风格泛化的发音解缠链接：https://arxiv.org/abs/2306.15304
*与cs.SD语音【21】为同一篇
作者：Yahuan Cong,Haoyu Zhang,Haopeng Lin,Shichao Liu,Chunfeng Wang,Yi Ren,Xiang Yin,Zejun Ma机构：ByteDance备注：Accepted by INTERSPEECH 2023摘要：跨语言音色和风格可泛化的文本到语音（TTS）旨在合成具有特定参考音色或风格的语音，该参考音色或风格从未在目标语言中训练过。它面临以下挑战：1）音色和发音是相关的，因为特定说话者的多语言语音通常难以获得;（2）语体和语音是混合的，因为言语语体既有语言不可知的部分，也有语言特有的部分。为了应对这些挑战，我们提出了GenerTTS，主要包括以下工作：1）我们精心设计了一个基于HuBERT的信息瓶颈，以解开音色和发音 风格的纠缠; 2）通过最小化风格与语言之间的互信息，去除风格嵌入中的语言相关信息。实验表明，GenerTTS优于基线系统的风格相似性和发音准确性，并实现跨语言的音色和风格的泛化。摘要：Cross-lingual timbre and style generalizable text-to-speech (TTS) aims to synthesize speech with a specific reference timbre or style that is never trained in the target language. It encounters the following challenges: 1) timbre and pronunciation are correlated since multilingual speech of a specific speaker is usually hard to obtain; 2) style and pronunciation are mixed because the speech style contains language-agnostic and language-specific parts. To address these challenges, we propose GenerTTS, which mainly includes the following works: 1) we elaborately design a HuBERT-based information bottleneck to disentangle timbre and pronunciation style; 2) we minimize the mutual information between style and language to discard the language-specific information in the style embedding. The experiments indicate that GenerTTS outperforms baseline systems in terms of style similarity and pronunciation accuracy, and enables cross-lingual timbre and style generalization.
【17】 Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition标题：用于老年人和节律障碍语音识别的一致性ASR系统的超参数自适应链接：https://arxiv.org/abs/2306.15265
作者：Tianzi Wang,Shoukang Hu,Jiajun Deng,Zengrui Jin,Mengzhe Geng,Yi Wang,Helen Meng,Xunying Liu机构：The Chinese University of Hong Kong, China, Nanyang Technological University, Singapore备注：5 pages, 3 figures, 3 tables, accepted by Interspeech2023摘要：迄今为止，由于数据稀缺，对障碍和老年人语音的自动识别仍然是极具挑战性的任务。参数微调通常用于利用大量非老化和健康的语音预训练模型，而神经架构超参数使用专家知识设置并保持不变。本文研究了Conformer ASR系统的超参数自适应，这些系统在被适配到DementiaBank老年人和UASpeech构音障碍语音数据集之前在Librisepeech语料库上进行了预训练。实验结果表明，超参数自适应产生的字错误率（WER）减少0.45%和0.67%以上的参数，只有微调DBank和UASpeech任务。一个直观的相关性之间的性能改善超参数域自适应和相对的话语长度比之间的源和目标域数据。摘要：Automatic recognition of disordered and elderly speech remains highly challenging tasks to date due to data scarcity. Parameter fine-tuning is often used to exploit the large quantities of non-aged and healthy speech pre-trained models, while neural architecture hyper-parameters are set using expert knowledge and remain unchanged. This paper investigates hyper-parameter adaptation for Conformer ASR systems that are pre-trained on the Librispeech corpus before being domain adapted to the DementiaBank elderly and UASpeech dysarthric speech datasets. Experimental results suggest that hyper-parameter adaptation produced word error rate (WER) reductions of 0.45% and 0.67% over parameter-only fine-tuning on DBank and UASpeech tasks respectively. An intuitive correlation is found between the performance improvements by hyper-parameter domain adaptation and the relative utterance length ratio between the source and target domain data.
【18】 Wespeaker baselines for VoxSRC2023标题：VoxSRC2023的Wespeaker基准链接：https://arxiv.org/abs/2306.15161
*与cs.SD语音【22】为同一篇
作者：Shuai Wang,Chengdong Liang,Xu Xiang,Bing Han,Zhengyang Chen,Hongji Wang,Wen Ding机构：Wespeaker Team, WeNet Open Source Community摘要：本报告展示了使用wespeaker工具包进行VoxSRC2023挑战所取得的成果。我们的目标是为参与者，特别是那些经验有限的人，提供清晰而直接的指导方针来开发他们的初始系统。通过结构良好的食谱和强大的结果，我们希望为所有感兴趣的人提供一个方便和足够好的起点。在本报告中，我们描述了使用预训练模型在VoxSRC2023开发集上获得的结果，您可以检查CodaLab评估服务器以获得评估集上的结果。摘要：This report showcases the results achieved using the wespeaker toolkit for the VoxSRC2023 Challenge. Our aim is to provide participants, especially those with limited experience, with clear and straightforward guidelines to develop their initial systems. Via well-structured recipes and strong results, we hope to offer an accessible and good enough start point for all interested individuals. In this report, we describe the results achieved on the VoxSRC2023 dev set using the pretrained models, you can check the CodaLab evaluation server for the results on the evaluation set.
【19】 Large-scale unsupervised audio pre-training for video-to-speech synthesis标题：视频到语音合成的大规模无监督音频预训练链接：https://arxiv.org/abs/2306.15464
*与cs.SD语音【14】为同一篇
作者：Triantafyllos Kefalas,Yannis Panagakis,Maja Pantic机构：uk)Triantafyllos Kefalas and Maja Pantic are with the Department of Comput-ing, UKYannis Panagakis is with the Department of Informatics and Telecommu-nications, University of Athens备注：Submitted to IEEE摘要：视频到语音合成是从说话者的无声视频重构语音信号的任务。迄今为止，大多数已建立的方法涉及两步过程，其中首先提取来自视频的中间表示（诸如频谱图），然后将其传递到声码器以产生原始音频。一些最近的工作集中在端到端合成上，由此原始音频和任何中间表示的生成被联合执行。所有这些方法都涉及对几乎完全来自视听数据集的数据进行训练，即每个音频样本具有相应的视频样本。这排除了使用可能不具有对应的视觉模态（例如，视觉模态）的丰富的仅音频数据集。有声读物、无线电播客、语音识别数据集等），以及音频机器学习社区多年来开发的纯音频架构。在本文中，我们建议在24 kHz的超过3,500小时的音频数据上训练编码器-解码器模型，然后使用预训练的解码器来初始化音频解码器，以进行视频到语音合成任务。预训练步骤仅使用音频样本，并且不需要来自其他模态（视觉、文本）的标签或对应样本。我们证明，这个预训练步骤提高了重建的语音，这是一个未探索的方式来提高质量的发生器在一个跨模态的任务，而只需要从一个模态的样本。我们进行实验，使用原始音频和梅尔频谱图作为目标输出和基准我们的模型与现有的工作。摘要：Video-to-speech synthesis is the task of reconstructing the speech signal from a silent video of a speaker. Most established approaches to date involve a two-step process, whereby an intermediate representation from the video, such as a spectrogram, is extracted first and then passed to a vocoder to produce the raw audio. Some recent work has focused on end-to-end synthesis, whereby the generation of raw audio and any intermediate representations is performed jointly. All such approaches involve training on data from almost exclusively audio-visual datasets, i.e. every audio sample has a corresponding video sample. This precludes the use of abundant audio-only datasets which may not have a corresponding visual modality (e.g. audiobooks, radio podcasts, speech recognition datasets etc.), as well as audio-only architectures that have been developed by the audio machine learning community over the years. In this paper we propose to train encoder-decoder models on more than 3,500 hours of audio data at 24kHz, and then use the pre-trained decoders to initialize the audio decoders for the video-to-speech synthesis task. The pre-training step uses audio samples only and does not require labels or corresponding samples from other modalities (visual, text). We demonstrate that this pre-training step improves the reconstructed speech and that it is an unexplored way to improve the quality of the generator in a cross-modal task while only requiring samples from one of the modalities. We conduct experiments using both raw audio and mel spectrograms as target outputs and benchmark our models with existing work.
【20】 RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Music标题：RMVPE：一种稳健的复调音乐基音估计模型链接：https://arxiv.org/abs/2306.15412
*与cs.SD语音【15】为同一篇
作者：Haojie Wei,Xueke Cao,Tangpeng Dan,Yueguo Chen机构：School of Information, Renmin University of China, Beijing, China, Institute of Information Science, Beijing Jiaotong University, Beijing, China备注：This paper has been accepted by INTERSPEECH 2023摘要：基音是音乐音频处理中的一个重要的高级特征。然而，在复调音乐中，由于伴奏的存在，提取音高更具挑战性。为了消除伴奏的影响，大多数以前的方法采用音乐源分离模型，以获得干净的人声从复调音乐之前预测音高。因此，基音估计的性能受到音乐源分离模型的影响。为了解决这个问题，并直接从复调音乐中提取音高，我们提出了一个强大的模型RMVPE。该模型能有效地提取复调音乐中的隐含特征，准确地预测出复调音乐中的音高。实验结果表明，RMVPE的原始音高精度（RPA）和原始色度精度（RCA）方面的优越性。此外，用不同类型的噪声进行的实验表明，RMVPE在所有信噪比（SNR）水平上都是鲁棒的。RMVPE的代码可在此URL中获得。https: github.com Dream-High RMVPE.摘要：Vocal pitch is an important high-level feature in music audio processing. However, extracting vocal pitch in polyphonic music is more challenging due to the presence of accompaniment. To eliminate the influence of the accompaniment, most previous methods adopt music source separation models to obtain clean vocals from polyphonic music before predicting vocal pitches. As a result, the performance of vocal pitch estimation is affected by the music source separation models. To address this issue and directly extract vocal pitches from polyphonic music, we propose a robust model named RMVPE. This model can extract effective hidden features and accurately predict vocal pitches from polyphonic music. The experimental results demonstrate the superiority of RMVPE in terms of raw pitch accuracy (RPA) and raw chroma accuracy (RCA). Additionally, experiments conducted with different types of noise show that RMVPE is robust across all signal-to-noise ratio (SNR) levels. The code of RMVPE is available at this URL. https: github.com Dream-High RMVPE.
【21】 Multi-perspective Information Fusion Res2Net with RandomSpecmix for Fake Speech Detection标题：基于RandomspecMix的多视角信息融合Res2Net假语音检测链接：https://arxiv.org/abs/2306.15389
*与cs.SD语音【16】为同一篇
作者：Shunbo Dong,Jun Xue,Cunhang Fan,Kang Zhu,Yujie Chen,Zhao Lv机构：Anhui Province Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University (AHU), Jiulong Road, Hefei, China备注：Accepted by DADA2023摘要：在本文中，我们提出了多视角信息融合（MPIF）Res2Net与随机频谱混合的假语音检测（FSD）。该系统的主要目的是提高模型的能力，学习准确的伪造信息，FSD任务在低质量的情况下。随机Spectmix是一种数据增强方法，其任务是提高模型的泛化能力，增强模型定位判别信息的能力。Spectmix在不引入其他数据的情况下，将频谱图的频率维度信息剪切粘贴在同一批样本中，有助于模型定位真正有用的信息。同时，我们随机选择样本进行扩增，以减少数据扩增直接改变所有数据的影响。一旦达到帮助模型定位信息的目的，减少不必要的信息也很重要。MPIF-Res2Net的作用是减少冗余干扰信息。从单一角度看的欺骗信息总是相似的，因此学习这种相似信息的模型会产生冗余的欺骗线索，干扰真正的区分信息。提出的MPIF-Res2Net融合了不同角度的信息，使模型学习的信息更加多样，从而减少了相似信息造成的冗余，避免了对区分信息学习的干扰。ASVspoof 2021 LA数据集的结果证明了我们提出的方法的有效性，EER和min-tDCF分别为3.29%和0.2557。摘要：In this paper, we propose the multi-perspective information fusion (MPIF) Res2Net with random Specmix for fake speech detection (FSD). The main purpose of this system is to improve the model's ability to learn precise forgery information for FSD task in low-quality scenarios. The task of random Specmix, a data augmentation, is to improve the generalization ability of the model and enhance the model's ability to locate discriminative information. Specmix cuts and pastes the frequency dimension information of the spectrogram in the same batch of samples without introducing other data, which helps the model to locate the really useful information. At the same time, we randomly select samples for augmentation to reduce the impact of data augmentation directly changing all the data. Once the purpose of helping the model to locate information is achieved, it is also important to reduce unnecessary information. The role of MPIF-Res2Net is to reduce redundant interference information. Deceptive information from a single perspective is always similar, so the model learning this similar information will produce redundant spoofing clues and interfere with truly discriminative information. The proposed MPIF-Res2Net fuses information from different perspectives, making the information learned by the model more diverse, thereby reducing the redundancy caused by similar information and avoiding interference with the learning of discriminative information. The results on the ASVspoof 2021 LA dataset demonstrate the effectiveness of our proposed method, achieving
【22】 3D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglement标题：3D-Speaker：一个大规模的多设备、多距离、多方言的语音表示解缠语料库链接：https://arxiv.org/abs/2306.15354
*与cs.SD语音【17】为同一篇
作者：Siqi Zheng,Luyao Cheng,Yafeng Chen,Hui Wang,Qian Chen机构：DAMO Academy, Alibaba Group摘要：从语音话语中提取不相关信息是语音学界的一个重要研究课题。不同的语音相关的任务集中在提取不同的语音表示，同时尽量减少其他不相关的信息的影响。我们提出了一个大规模的语音语料库，以促进语音表征去纠缠的研究。3D-Speaker包含超过10，000个扬声器，每个扬声器由位于不同距离的多个设备同时录制，并且一些扬声器正在讲多种方言。多维音频数据的受控组合产生语音表示纠缠的多样化混合的矩阵，从而激发有趣的方法来解开它们。3D-Speaker的多域性质也使其成为评估大型通用语音模型和域外学习和自监督学习的实验方法的合适资源。https: 3dspeaker.github.io摘要：Disentangling uncorrelated information in speech utterances is a crucial research topic within speech community. Different speech-related tasks focus on extracting distinct speech representations while minimizing the affects of other uncorrelated information. We present a large-scale speech corpus to facilitate the research of speech representation disentanglement. 3D-Speaker contains over 10,000 speakers, each of whom are simultaneously recorded by multiple Devices, locating at different Distances, and some speakers are speaking multiple Dialects. The controlled combinations of multi-dimensional audio data yield a matrix of a diverse blend of speech representation entanglement, thereby motivating intriguing methods to untangle them. The multi-domain nature of 3D-Speaker also makes it a suitable resource to evaluate large universal speech models and experiment methods of out-of-domain learning and self-supervised learning. https: 3dspeaker.github.io
【23】 TranssionADD: A multi-frame reinforcement based sequence tagging model for audio deepfake detection标题：TranssionADD：一种基于多帧强化的音频深度伪检测序列标注模型链接：https://arxiv.org/abs/2306.15212
*与cs.SD语音【18】为同一篇
作者：Jie Liu,Zhiba Su,Hui Huang,Caiyan Wan,Quanxiu Wang,Jiangli Hong,Benlai Tang,Fengjie Zhu机构：Department of AI Technology, Transsion摘要：由于端到端语音建模技术的最新进展，模仿和克隆用户的声音变得越来越可行。这导致区分真实音频段和伪造音频段的重大挑战。为了解决用户语音滥用和误用的问题，第二届音频Deepfake检测挑战赛（ADD 2023）旨在检测和分析deepfake语音。具体地，被称为操纵区域位置（RL）的轨道2旨在精确定位音频中的操纵区域的位置，其可以存在于真实的和生成的音频片段两者中。我们提出了我们的新的TranssionADD系统作为一个具有挑战性的问题，模型的鲁棒性和音频段离群值的跟踪竞争的解决方案。我们的系统提供了三个独特的贡献：1）我们将序列标记任务适应于音频深度伪造检测; 2）通过各种数据增强技术提高模型的泛化能力; 3）我们结合多帧检测（MFD）模块来克服由单个帧提供的有限表示，并使用隔离帧惩罚（IFP）损失来处理片段中的离群值。我们最好的提交在Track 2中获得了第二名，证明了我们提出的系统的有效性和鲁棒性。摘要：Thanks to recent advancements in end-to-end speech modeling technology, it has become increasingly feasible to imitate and clone a user s voice. This leads to a significant challenge in differentiating between authentic and fabricated audio segments. To address the issue of user voice abuse and misuse, the second Audio Deepfake Detection Challenge (ADD 2023) aims to detect and analyze deepfake speech utterances. Specifically, Track 2, named the Manipulation Region Location (RL), aims to pinpoint the location of manipulated regions in audio, which can be present in both real and generated audio segments. We propose our novel TranssionADD system as a solution to the challenging problem of model robustness and audio segment outliers in the trace competition. Our system provides three unique contributions: 1) we adapt sequence tagging task for audio deepfake detection; 2) we improve model generalization by various data augmentation techniques; 3) we incorporate multi-frame detection (MFD) module to overcome limited representation provided by a single frame and use isolated-frame penalty (IFP) loss to handle outliers in segments. Our best submission achieved 2nd place in Track 2, demonstrating the effectiveness and robustness of our proposed system.
【24】 A Deep Learning Architecture with Spatio-Temporal Focusing for Detecting Respiratory Anomalies标题：一种具有时空聚焦的呼吸异常检测深度学习结构链接：https://arxiv.org/abs/2306.14929
*与cs.SD语音【19】为同一篇
作者：Dat Ngo,Lam Pham,Huy Phan,Minh Tran,Delaram Jarchi机构：. School of Computer Science and Electronic Engineering, University of Essex, UK.备注：arXiv admin note: text overlap with arXiv:2303.04104摘要：本文提出了一种应用于从呼吸声记录中检测异常的深度学习系统。我们的系统首先使用连续小波变换进行音频特征提取。该变换将呼吸声输入转换成二维频谱图，其中呈现频谱和时间特征两者。然后，我们提出的深度学习架构受到基于Inception-residual的主干的启发，执行时空聚焦和多头注意机制来对呼吸异常进行分类。在这项工作中，我们评估了我们提出的模型的基准SPRSound（开源上海交通大学儿科呼吸声）数据库提出的IEEE BioCAS 2023挑战。关于通过平均分数和谐波分数之间的平均值计算的分数，我们的鲁棒系统在任务1 - 1、1 - 2、2 - 1和2 - 2中分别以0.810、0.667、0.744和0.608的分数实现了Top-1性能。摘要：This paper presents a deep learning system applied for detecting anomalies from respiratory sound recordings. Our system initially performs audio feature extraction using Continuous Wavelet transformation. This transformation converts the respiratory sound input into a two-dimensional spectrogram where both spectral and temporal features are presented. Then, our proposed deep learning architecture inspired by the Inception-residual-based backbone performs the spatial-temporal focusing and multi-head attention mechanism to classify respiratory anomalies. In this work, we evaluate our proposed models on the benchmark SPRSound (The Open-Source SJTU Paediatric Respiratory Sound) database proposed by the IEEE BioCAS 2023 challenge. As regards the Score computed by an average between the average score and harmonic score, our robust system has achieved Top-1 performance with Scores of 0.810, 0.667, 0.744, and 0.608 in Tasks 1-1, 1-2, 2-1, and 2-2, respectively. 
每日语音速递