[Paper 리뷰] Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance
Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance Classifier guidance를 활용하여 target speaker의 transcript 없이 고품질의 text-to-speech를 수행할 수 있음 Guided-TTS Classifier guidance를 위해 개별적으로 train된 phoneme classifier와 unconditional diffusion model을 결합 Unconditional diffusion model은 untranscribed speech data로부터 context 없이 음성을 생성하는 방법을 학습 Phoneme classifier를 사용하여 diffusion model의 generati..
[Paper 리뷰] HierSpeech: Bridging the Gap between Text and Speech by Hierarchical Variational Inference using Self-Supervised Representations for Speech Synthesis
HierSpeech: Bridging the Gap between Text and Speech by Hierarchical Variational Inference using Self-Supervised Representations for Speech Synthesis Text로부터 raw waveform을 직접 생성하는 end-to-end pipeline은 고품질의 합성이 가능하지만, mispronunciation이나 over-smoothing 문제가 종종 발생함 HierSpeech 고품질의 end-to-end text-to-speech를 위해 self-supervised speech representation을 활용하는 hierarchical variational autoencoder 구조 Self-s..
[Paper 리뷰] Pruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speech
Pruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speech Personalized speech generation을 위해서는 target speaker의 limited data를 사용해서 Text-to-Speech를 수행해야 함 이를 위해 text-to-speech 모델은 out-of-domain data에 대해 amply generalize 되어야 함 Self-Attention Pruning Generalization을 위해 text-to-speech 모델의 transformer에 대해 spase attention을 통한 pruning을 적용 Attention weight가 threshold 보다 낮은 self-attention layer에서 ..
[Paper 리뷰] ZET-Speech: Zero-Shot Adaptive Emotion-Controllable Text-to-Speech with Diffusion and Style-based Models
ZET-Speech: Zero-Shot Adaptive Emotion-Controllable Text-to-Speech Synthesis with Diffusion and Style-based Models Emotional Text-to-Speech는 natural 하고 emotional한 음성을 합성할 수 있음 BUT, 기존 방식들은 unseen speaker에 대한 generalization 없이 seen speaker만을 대상으로 함 ZET-Speech 짧은 speech segment와 target emotion label을 사용하여 any-speaker zero-shot adaptive text-to-speech 수행 Zero-shot adaptive model이 emotional speech를 ..
[Paper 리뷰] JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech
JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to SpeechText-to-Speech는 2-stage 방식이나 개별적으로 training 된 모델의 cascade로 학습됨BUT, training pipeline은 최적의 성능을 위해서 fine-tuning이나 speech-text alignment를 요구함JETSSimplified pipeline을 구성해 개별적으로 학습된 모델들보다 뛰어난 성능을 발휘하는 end-to-end 모델을 제시Alignment module을 사용하여 FastSpeech2와 HiFi-GAN을 jointly trainingAlignment learning objective를 채택하여 external al..
[Paper 리뷰] StyleTTS2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
StyleTTS2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language ModelsHuman-level text-to-speech를 위해 large speech language model (SLM)을 활용할 수 있음StyleTTS2Diffusion model을 통해 style을 latent random variable로 모델링하여 reference speech 없이 text에 적합한 style을 생성End-to-End training을 위해 differentiable duration modeling이 가능한 discriminator를 도입하고 large pre..