[Paper 리뷰] CrossSpeech: Speaker-Independent Acoustic Representation for Cross-Lingual Speech Synthesis
CrossSpeech: Speaker-Independent Acoustic Representation for Cross-Lingual Speech SynthesisCross-lingual Text-to-Speech 성능은 여전히 intra-lingual 성능보다 떨어짐CrossSpeechSpeaker와 language information의 disentangling을 acoustic feature space level에서 효과적으로 disentangling 하여 cross-lingual text-to-speech 성능을 향상이를 위해 Speaker-Independent Generator와 Speaker-Dependent Generator를 도입하고 각 information을 개별적으로 처리함으로써 dis..
[Paper 리뷰] DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANsDenoising Diffusion Probabilistic Model (DDPM)은 음성 합성에서 우수한 성능으로 보이고 있지만, 높은 sampling cost의 문제가 있음DiffGAN-TTSDenoising distribution을 근사하기 위해 adversarially-trained expressive model을 채택한 denoising diffusion generative adversarial network (GAN)을 기반으로 함추가적으로 추론 속도를 더욱 향상하기 위해 active shallow diffusion mechanism을 도입Tw..
[Paper 리뷰] CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models
CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency ModelsText-to-Speech에서 diffusion model을 사용하면 high-fidelity의 음성을 합성할 수 있지만 multi-step sampling으로 인해 real-time synthesis에는 한계가 있음한편으로 GAN과 diffusion model을 결합하여 denoising distribution을 근사하는 방식으로 추론 속도를 개선할 수 있지만, adversarial training으로 인해 모델 수렴의 어려움이 있음CM-TTSConsistency Model (CM)을 기반으로 advers..
[Paper 리뷰] DurIAN-E2: Duration Informed Attention Network with Adaptive Variational Autoencoder and Adversarial Learning for Expressive Text-to-Speech Synthesis
DurIAN-E2: Duration Informed Attention Network with Adaptive Variational Autoencoder and Adversarial Learning for Expressive Text-to-Speech SynthesisExpressive, high-fidelity text-to-speech를 위해 duration informed attention model을 고려할 수 있음DurIAN-E2Multiple stacked SwishRNN-based Transformer block을 linguistic encoder로 채택하고 Style-Adaptive Instance Normalization layer를 frame-level encoder에 추가하여 expre..
[Paper 리뷰] DETS: End-to-End Single-Stage Text-to-Speech via Hierarchical Diffusion GAN Models
DETS: End-to-End Single-Stage Text-to-Speech via Hierarchical Diffusion GAN ModelsEnd-to-End text-to-speech는 여전히 naturalness와 prosody diversity 측면에서 한계가 있음DETSHierarchical denoising diffusion GAN을 도입한 end-to-end frameworkDenoising distribution을 모델링하기 위해 non-Gaussian multi-modal function을 채택하여 다양한 pitch와 rhythm을 반영할 수 있는 one-to-many relationship을 학습논문 (ICASSP 2024) : Paper Link1. IntroductionText..
[Paper 리뷰] StyleSpeech: Self-Supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesis
StyleSpeech: Self-Supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech SynthesisAudiobook을 위한 음성 합성은 generalized architecture와 training data의 unbalanced style distribution으로 인해 한계가 있음StyleSpeechExpressive audiobook synthesis를 위해 VQ-VAE-based pre-training을 통한 self-supervised style enhancing method를 적용Text style encoder는 large-scale unlabeled text-only data로 p..