[Paper 리뷰] X-Singer: Code-Mixed Singing Voice Synthesis via Cross-Lingual Learning
X-Singer: Code-Mixed Singing Voice Synthesis via Cross-Lingual LearningSinging Voice Synthesis는 여전히 musical score의 annotation에 의존적이고 code-mixed singing voice를 생성하는 데는 한계가 있음X-SingerPhoneme annotation이 없는 code-mixed lyrics로 구성된 music score를 처리하는 music score encoder를 도입- Music score encoder는 code-mixed lyrics를 encode하기 위해 language code-switching을 채택하고, phoneme annotation에 대한 의존성을 줄이기 위해 mixture al..
[Paper 리뷰] VISinger2: High-Fidelity End-to-End Singing Voice Synthesis Enhanced by Digital Signal Processing Synthesizer
VISinger2: High-Fidelity End-to-End Singing Voice Synthesis Enhanced by Digital Signal Processing SynthesizerSinging Voice Synthesis에서 VISinger는 우수한 합성 성능을 달성했지만 다음의 한계점이 존재함- Text-to-Phase problem, Glitches, Low sampling rateVISinger2Digital signal processing synthesizer를 통해 VISinger의 latent representation $z$로부터 periodic/aperiodic signal을 생성- Phase information 없이 latent representation을 추출하도록 p..
[Paper 리뷰] PeriodSinger: Integrating Periodic and Aperiodic Variational Autoencoders for Natural-Sounding End-to-End Singing Voice Synthesis
PeriodSinger: Integrating Periodic and Aperiodic Variational Autoencoders for Natural-Sounding End-to-End Singing Voice Synthesis자연스러운 waveform을 합성하기 위해서는 deterministic pitch conditioning으로 인한 one-to-many 문제를 해결해야 함PeriodSingerPeriodic/aperiodic component에 대한 variational autoencoder를 활용Note boundary 내에서 monotonic alignment search를 통해 phoneme alignment를 추정함으로써 external aligner에 대한 의존성을 제거논문 (INTE..
[Paper 리뷰] MakeSinger: A Semi-Supervised Training Method for Data-Efficient Singing Voice Synthesis via Classifier-free Diffusion Guidance
MakeSinger: A Semi-Supervised Training Method for Data-Efficient Singing Voice Synthesis via Classifier-free Diffusion GuidanceSinging voice synthesis를 위해 semi-supervised training을 활용할 수 있음MakeSingerLabeling에 관계없이 모든 speech, singing voice data에서 diffusion-based model을 trainingDual guiding mechanism을 통해 maske input의 score를 추정하여 reverse diffusion step에 대한 text/pitch guidance를 제공논문 (INTERSPEECH 202..
[Paper 리뷰] TokSing: Singing Voice Synthesis based on Discrete Tokens
TokSing: Singing Voice Synthesis based on Discrete TokensSelf-supervised learning model에서 추출된 discrete token을 활용하여 singing voice synthesis의 성능을 향상할 수 있음TokSingFlexible token blending을 제공하는 token formulator를 갖춘 discrete-based singing voice synthesis modelMelody signal을 discrete token과 integrate 하고 musical encoder에 melody enhancement strategy를 도입논문 (INTERSPEECH 2024) : Paper Link1. IntroductionSin..
[Paper 리뷰] Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language PromptSinging Voice Synthesis는 여전히 style attribute를 explicitly control 하는 것이 어려움Prompt-SingerGender, vocal range, volume 등을 natural language prompt로 control하는 singing voice synthesis 모델Multi-scale hierarchy를 가지는 decoder-only transformer를 기반으로 melodic accuracy를 유지하면서 text-conditioned vocal range control이 가능한 range-melody de..