[Paper 리뷰] DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech
DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-SpeechCross-lingual text-to-speech는 speaker timbre를 정확하게 retain 하면서 nativeness를 반영하는 것이 어려움DSE-TTSMel-spectrogram 보다 더 적은 speaker information을 포함하는 vector-quantized acoustic feature를 활용해당 acoustic feature를 기반으로 speaking style을 반영하는 Dual Speaker Embedding을 도입- 한 embedding은 linguistic speaking stlye을 학습하기 위해 acoustic model에 전달되고,- 다른 embedd..
[Paper 리뷰] AutoTTS: End-to-End Text-to-Speech Synthesis through Differentiable Duration Modeling
AutoTTS: End-to-End Text-to-Speech through Differentiable Duration ModelingText-to-Speech 모델은 일반적으로 external aligner가 필요하고, decoder와 jointly train 되지 않으므로 최적화의 한계가 있음AutoTTSInput, output sequence 간의 monotonic alignment를 학습하기 위해 differentiable duration method를 도입Expectation에서 stochastic process를 최적화하는 soft-duration mechanism을 기반으로 하여 direct text-to-waveform synthesis 모델을 구축추가적으로 adversarial train..
[Paper 리뷰] CrossSpeech: Speaker-Independent Acoustic Representation for Cross-Lingual Speech Synthesis
CrossSpeech: Speaker-Independent Acoustic Representation for Cross-Lingual Speech SynthesisCross-lingual Text-to-Speech 성능은 여전히 intra-lingual 성능보다 떨어짐CrossSpeechSpeaker와 language information의 disentangling을 acoustic feature space level에서 효과적으로 disentangling 하여 cross-lingual text-to-speech 성능을 향상이를 위해 Speaker-Independent Generator와 Speaker-Dependent Generator를 도입하고 각 information을 개별적으로 처리함으로써 dis..
[Paper 리뷰] DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANsDenoising Diffusion Probabilistic Model (DDPM)은 음성 합성에서 우수한 성능으로 보이고 있지만, 높은 sampling cost의 문제가 있음DiffGAN-TTSDenoising distribution을 근사하기 위해 adversarially-trained expressive model을 채택한 denoising diffusion generative adversarial network (GAN)을 기반으로 함추가적으로 추론 속도를 더욱 향상하기 위해 active shallow diffusion mechanism을 도입Tw..
[Paper 리뷰] CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models
CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency ModelsText-to-Speech에서 diffusion model을 사용하면 high-fidelity의 음성을 합성할 수 있지만 multi-step sampling으로 인해 real-time synthesis에는 한계가 있음한편으로 GAN과 diffusion model을 결합하여 denoising distribution을 근사하는 방식으로 추론 속도를 개선할 수 있지만, adversarial training으로 인해 모델 수렴의 어려움이 있음CM-TTSConsistency Model (CM)을 기반으로 advers..
[Paper 리뷰] DurIAN-E2: Duration Informed Attention Network with Adaptive Variational Autoencoder and Adversarial Learning for Expressive Text-to-Speech Synthesis
DurIAN-E2: Duration Informed Attention Network with Adaptive Variational Autoencoder and Adversarial Learning for Expressive Text-to-Speech SynthesisExpressive, high-fidelity text-to-speech를 위해 duration informed attention model을 고려할 수 있음DurIAN-E2Multiple stacked SwishRNN-based Transformer block을 linguistic encoder로 채택하고 Style-Adaptive Instance Normalization layer를 frame-level encoder에 추가하여 expre..