今日论文合集：cs.SD语音2篇，eess.AS音频处理2篇。
【1】 Multi-Genre Music Transformer -- Composing Full Length Musical Piece
标题：多体裁音乐Transformer--创作长篇音乐作品
链接：https://arxiv.org/abs/2301.02385
作者：Abhinav Kaushal Keshari机构：Multi-Genre Music Transformer - Composing Full Length Musical PieceAbhinav Kaushal Keshari (Purdue University)AbstractIn the task of generating music摘要：在生成音乐的任务中，艺术因素起着很大的作用，对AI来说是一个很大的挑战。以前的工作涉及对抗性训练来产生新的音乐作品和建模的兼容性的多样性音乐（节拍，速度，音乐茎）证明了很好的例子学习这项任务。尽管这仅限于生成混搭或从节奏和键分布中学习特征以产生类似的模式。复合词Transformer能够将音乐生成任务表示为涉及由复合词定义的音乐事件的序列生成挑战。这些音乐事件给出了一个更准确的描述音符进展，和弦变化，和声和艺术因素。该项目的目标是实现一个多流派Transformer，它通过更适应性的学习过程来学习制作音乐作品，该过程涉及更具挑战性的任务，其中还考虑了作曲的流派或形式。我们建立了一个多体裁复合词数据集，实现了一个线性Transformer，并在这个数据集上训练。我们称之为多流派Transformer，这是能够产生全长新的音乐作品，这是多样的和可比的原始轨道。该模型的训练速度比所讨论的其他模型快2-5倍。摘要：In the task of generating music, the art factor plays a big role and is a great challenge for AI. Previous work involving adversarial training to produce new music pieces and modeling the compatibility of variety in music (beats, tempo, musical stems) demonstrated great examples of learning this task. Though this was limited to generating mashups or learning features from tempo and key distributions to produce similar patterns. Compound Word Transformer was able to represent music generation task as a sequence generation challenge involving musical events defined by compound words. These musical events give a more accurate description of notes progression, chord change, harmony and the art factor. The objective of the project is to implement a Multi-Genre Transformer which learns to produce music pieces through more adaptive learning process involving more challenging task where genres or form of the composition is also considered. We built a multi-genre compound word dataset, implemented a linear transformer which was trained on this dataset. We call this Multi-Genre Transformer, which was able to generate full length new musical pieces which is diverse and comparable to original tracks. The model trains 2-5 times faster than other models discussed.
【2】 Singing voice synthesis based on frame-level sequence-to-sequence models considering vocal timing deviation
标题：基于考虑发声时序偏差的帧级序列到序列模型的歌唱语音合成
链接：https://arxiv.org/abs/2301.02262
作者：Miku Nishihara,Yukiya Hono,Kei Hashimoto,Yoshihiko Nankaku,Keiichi Tokuda机构：Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan备注：5 pages, 4 figures, submitted to ICASSP 2023摘要：提出了一种基于帧级序列模型的考虑人声时序偏差的歌唱声合成方法。在SVS中，考虑到实际发声定时和音符开始定时之间存在差异，将歌唱定时与由乐谱表示的时间结构同步是必要的。在许多SVS系统（包括我们以前的工作）中，基于由外部对准器获得的音素边界将音素级得分特征转换为帧级得分特征以考虑声音定时偏差。因此，在该系统中，声音质量受到对准器精度的影响。为了缓解这个问题，我们引入了一个具有帧级特征的注意机制。在所提出的系统中，注意机制吸收了音素边界的对齐误差。此外，我们评估了在没有对齐器的情况下，基于乐谱的启发式规则定义的伪音素边界的系统。实验结果验证了该系统的有效性。摘要：This paper proposes singing voice synthesis (SVS) based on frame-level sequence-to-sequence models considering vocal timing deviation. In SVS, it is essential to synchronize the timing of singing with temporal structures represented by scores, taking into account that there are differences between actual vocal timing and note start timing. In many SVS systems including our previous work, phoneme-level score features are converted into frame-level ones on the basis of phoneme boundaries obtained by external aligners to take into account vocal timing deviations. Therefore, the sound quality is affected by the aligner accuracy in this system. To alleviate this problem, we introduce an attention mechanism with frame-level features. In the proposed system, the attention mechanism absorbs alignment errors in phoneme boundaries. Additionally, we evaluate the system with pseudo-phoneme-boundaries defined by heuristic rules based on musical scores when there is no aligner. The experimental results show the effectiveness of the proposed system.【1】 Singing voice synthesis based on frame-level sequence-to-sequence models considering vocal timing deviation
标题：基于考虑发声时序偏差的帧级序列到序列模型的歌唱语音合成
链接：https://arxiv.org/abs/2301.02262
*与cs.SD语音【2】为同一篇
作者：Miku Nishihara,Yukiya Hono,Kei Hashimoto,Yoshihiko Nankaku,Keiichi Tokuda机构：Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan备注：5 pages, 4 figures, submitted to ICASSP 2023摘要：提出了一种基于帧级序列模型的考虑人声时序偏差的歌唱声合成方法。在SVS中，考虑到实际发声定时和音符开始定时之间存在差异，将歌唱定时与由乐谱表示的时间结构同步是必要的。在许多SVS系统（包括我们以前的工作）中，基于由外部对准器获得的音素边界将音素级得分特征转换为帧级得分特征以考虑声音定时偏差。因此，在该系统中，声音质量受到对准器精度的影响。为了缓解这个问题，我们引入了一个具有帧级特征的注意机制。在所提出的系统中，注意机制吸收了音素边界的对齐误差。此外，我们评估了在没有对齐器的情况下，基于乐谱的启发式规则定义的伪音素边界的系统。实验结果验证了该系统的有效性。摘要：This paper proposes singing voice synthesis (SVS) based on frame-level sequence-to-sequence models considering vocal timing deviation. In SVS, it is essential to synchronize the timing of singing with temporal structures represented by scores, taking into account that there are differences between actual vocal timing and note start timing. In many SVS systems including our previous work, phoneme-level score features are converted into frame-level ones on the basis of phoneme boundaries obtained by external aligners to take into account vocal timing deviations. Therefore, the sound quality is affected by the aligner accuracy in this system. To alleviate this problem, we introduce an attention mechanism with frame-level features. In the proposed system, the attention mechanism absorbs alignment errors in phoneme boundaries. Additionally, we evaluate the system with pseudo-phoneme-boundaries defined by heuristic rules based on musical scores when there is no aligner. The experimental results show the effectiveness of the proposed system.
【2】 Multi-Genre Music Transformer -- Composing Full Length Musical Piece
标题：多体裁音乐Transformer--创作长篇音乐作品
链接：https://arxiv.org/abs/2301.02385
*与cs.SD语音【1】为同一篇
作者：Abhinav Kaushal Keshari机构：Multi-Genre Music Transformer - Composing Full Length Musical PieceAbhinav Kaushal Keshari (Purdue University)AbstractIn the task of generating music摘要：在生成音乐的任务中，艺术因素起着很大的作用，对AI来说是一个很大的挑战。以前的工作涉及对抗性训练来产生新的音乐作品和建模的兼容性的多样性音乐（节拍，速度，音乐茎）证明了很好的例子学习这项任务。尽管这仅限于生成混搭或从节奏和键分布中学习特征以产生类似的模式。复合词Transformer能够将音乐生成任务表示为涉及由复合词定义的音乐事件的序列生成挑战。这些音乐事件给出了一个更准确的描述音符进展，和弦变化，和声和艺术因素。该项目的目标是实现一个多流派Transformer，它通过更适应性的学习过程来学习制作音乐作品，该过程涉及更具挑战性的任务，其中还考虑了作曲的流派或形式。我们建立了一个多体裁复合词数据集，实现了一个线性Transformer，并在这个数据集上训练。我们称之为多流派Transformer，这是能够产生全长新的音乐作品，这是多样的和可比的原始轨道。该模型的训练速度比所讨论的其他模型快2-5倍。摘要：In the task of generating music, the art factor plays a big role and is a great challenge for AI. Previous work involving adversarial training to produce new music pieces and modeling the compatibility of variety in music (beats, tempo, musical stems) demonstrated great examples of learning this task. Though this was limited to generating mashups or learning features from tempo and key distributions to produce similar patterns. Compound Word Transformer was able to represent music generation task as a sequence generation challenge involving musical events defined by compound words. These musical events give a more accurate description of notes progression, chord change, harmony and the art factor. The objective of the project is to implement a Multi-Genre Transformer which learns to produce music pieces through more adaptive learning process involving more challenging task where genres or form of the composition is also considered. We built a multi-genre compound word dataset, implemented a linear transformer which was trained on this dataset. We call this Multi-Genre Transformer, which was able to generate full length new musical pieces which is diverse and comparable to original tracks. The model trains 2-5 times faster than other models discussed.
机器翻译，仅供参考
每日语音速递