[Paper 리뷰] NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-Robust Expressive TTS
NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-Robust Expressive TTSExpressive text-to-speech는 다음의 어려움이 존재함- Reference audio에 background noise가 포함된 경우 highly dynamic prosody information을 추출하기 어려움- Unseen speaking style에 대한 generalization이 가능해야 함NoreSpeechKnowledge distillation을 통해 teacher model에서 noise-agnostic speaking style을 학습하는 diffusion model에 기반한 DiffStyle m..
[Paper 리뷰] GenerTTS: Pronunciation Disentanglement for Timbre and Style Generalization in Cross-Lingual Text-to-Speech
GenerTTS: Pronunciation Disentanglement for Timbre and Style Generalization in Cross-Lingual Text-to-SpeechCross-lingual text-to-speech는 다음의 어려움이 있음- Timbre, pronunciation은 서로 correlate 되어 있음- Speech style에는 language-agnostic, language-specific part가 포함되어 있음GenerTTSPronunciation/style과 timbre를 disentangle 하기 위해 HuBERT-based information bottleneck을 도입Language-specific information을 제거하기 위해 style, ..
[Paper 리뷰] MultiVerse: Efficient and Expressive Zero-Shot Multi-Task Text-to-Speech
MultiVerse: Efficient and Expressive Zero-Shot Multi-Task Text-to-SpeechZero-Shot Text-to-Speech를 위해서는 많은 training data가 필요하고 기존보다 cost 증가함MultiVerse기존의 data-driven method 보다 더 적은 training data를 사용하면서 zero-shot 환경에서 Text-to-Speech, Style transfer를 수행하는 multi-task modelSource-filter theory-based disentanglement를 활용하고 filter-related/source-related representation을 모델링하기 위한 prompt를 도입Prosody similar..
[Paper 리뷰] PL-TTS: A Generalizable Prompt-based Diffusion TTS Augmented by Large Language Model
PL-TTS: A Generalizable Prompt-based Diffusion TTS Augmented by Large Language ModelStyle-controlled Text-to-Speech를 위해 text style description을 사용할 수 있음PL-TTSLarge Language Model로 embed 된 prompt와 diffusion-based Text-to-Speech model을 결합추가적으로 합성 품질과 style controllability를 향상하기 위해 Large Language Model과 diffusion framework를 fine-tuning논문 (INTERSPEECH 2024) : Paper Link1. IntroductionControllable ex..
[Paper 리뷰] ClariTTS: Feature-ratio Normalization and Duration Stabilization for Code-Mixed Multi-Speaker Speech Synthesis
ClariTTS: Feature-ratio Normalization and Duration Stabilization for Code-Mixed Multi-Speaker Speech SynthesisText-to-Speech model에서 code-mixed text는 speaker-related feature에 source language에 대한 linguistic feature가 포함될 수 있으므로 unnatural accent를 생성할 수 있음ClariTTSFlow-based text-to-speech model에 Feature-ratio Normalized Affine Coupling Layer를 적용- Speaker와 linguistic feature를 disentangle 하여 target sp..
[Paper 리뷰] VoiceTailor: Lightweight Plug-In Adapter for Diffusion-based Personalized Text-to-Speech
VoiceTailor: Lightweight Plug-In Adapter for Diffusion-based Personalized Text-to-SpeechPre-trained diffusion-based model에 personalized adapter를 결합하여 parameter-efficient speaker adaptive Text-to-Speech를 수행할 수 있음VoiceTailorParameter-Efficient Adaptation을 위해 Low-Rank Adaptation을 활용하고 adapter를 pre-trained diffusion decoder의 pivotal module에 통합Few parameter 만으로 강력한 adaptation을 달성하기 위해 guidance techni..