[Paper 리뷰] DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors
DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific FactorsLarge-scale Latent Diffusion Model은 various modality에 대해 우수한 content generation 성능을 보여주고 있지만, text-to-speech에서는 phoneme, duration에 의존해야 하므로 scalability의 한계가 있음DiTTo-TTSDomain-specific factor를 제거한 Latent Diffusion Model 기반의 text-to-speech model기존 U-Net 대신 Diffusion Transformer를 채택하고 speech length predicto..
[Paper 리뷰] BnTTS: Few-Shot Speaker Adaptation in Low-Resource Setting
BnTTS: Few-Shot Adaptation in Low-Resource SettingLow-resource language에 대한 text-to-speech model이 필요함BnTTSXTTS architecture를 기반으로 하는 speaker adaptation-based text-to-speech modelLow-resource language의 phonetic, linguistic character를 반영하도록 multilingual pipeline에 integrate 함논문 (NAACL 2025) : Paper Link1. IntroductionText-to-Speech (TTS)에서 zero-shot, one-shot adaptation은 여전히 한계가 있음대표적인 zero-shot TT..
[Paper 리뷰] ProsodyFlow: High-Fidelity Text-to-Speech through Conditional Flow Matching and Prosody Modeling with Large Speech Language Models
ProsodyFlow: High-Fidelity Text-to-Speech through Conditional Flow Matching and Prosody Modeling with Large Speech Language ModelsText-to-Speech에서 diverse, natural prosody를 반영하는 것은 여전히 한계가 있음ProsodyFlowLarge self-supervised speech model과 conditional flow matching을 결합해 prosodic feature를 modelingSpeech LLM을 통해 acoustic feature를 추출하고 해당 feature를 prosody latent space에 mapping 한 다음, conditional flow ..
[Paper 리뷰] FluentTTS: Text-dependent Fine-grained Style Control for Multi-style TTS
FluentTTS: Text-dependent Fine-grained Style Control for Multi-style TTSNeural text-to-speech model은 local prosodic variation을 flexibly control 할 수 있어야 함FluentTTSUtterance-wise global style embedding을 condition으로 하여 각 text의 fundamental frequency $F0$를 예측함추가적으로 global utterance-wise embedding과 local $F0$ embedding을 input으로 사용하는 multi-style encoder를 통해 multi-style embedding을 추정함논문 (INTERSPEECH 202..
[Paper 리뷰] VoiceLDM: Text-to-Speech with Environmental Context
VoiceLDM: Text-to-Speech with Environmental ContextDescription prompt와 content prompt를 활용하여 audio를 생성할 수 있음- Description prompt는 environmental context를 전달하고 content prompt는 linguistic information을 제공함VoiceLDMLatent diffusion model을 기반으로 하는 text-to-audio model을 채택하고 additional content prompt를 conditional input으로 활용할 수 있도록 확장Contrastive Language-Audio Pretraining과 Whisper를 활용하여 manual annotation, ..
[Paper 리뷰] Flowtron: An Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis
Flowtron: An Autoregressive Flow-based Generative Network for Text-to-Speech SynthesisStyle transfer, speech variation을 향상하기 위해 autoregressive flow-based generative network를 활용할 수 있음FlowtronTraining data의 likelihood를 maximizing 하여 optimize 되고 simple, stable training을 지원Timbre, expressivity, accent를 modulate할 수 있는 latent space에 대한 invertible mapping을 학습논문 (ICLR 2021) : Paper Link1. Introduction최근..