[Paper 리뷰] Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
Efficient Speech Language Modeling via Energy Distance in Continuous Latent SpaceSpeech language model은 discretization으로 인한 한계가 있음SLEDSpeech waveform을 continuous latent representation의 sequence로 encodingEnergy distance objective를 사용하여 autoregressive modeling을 수행논문 (NeurIPS 2025) : Paper Link1. IntroductionSpeech audio는 integer/floating-point range내의 value를 가지는 lengthy sampling point sequence로 re..
[Paper 리뷰] Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance
Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free GuidanceAutoregressive speech token generation model은 hallucination과 undesired vocalization의 문제가 있음Koel-TTSPreference alignment와 Classifier Free Guidance를 활용하여 Language Model의 contextual adherence를 향상특히 speech recognition model에서 derive 된 automatic metric을 사용하여 model output을 rank 하고 conditional, uncondi..
[Paper 리뷰] EmoVoice: LLM-based Emotional Text-to-Speech Model with Freestyle Text Prompting
EmoVoice: LLM-based Emotional Text-to-Speech Model with Freestyle Text PromptingText-to-Speech model은 여전히 emotional expression 측면에서 한계가 있음EmoVoiceLarge Language Model을 활용하여 fine-grained freestyle natural language emotion control을 지원Phoneme token과 audio token을 parallel output 하여 content consistency를 향상논문 (MM 2025) : Paper Link1. IntroductionEmotion-contorllable Text-to-Speech (TTS) model은 emotion..
[Paper 리뷰] PALLE: Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesis
PALLE: Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech SynthesisZero-Shot Text-to-Speech에서 autoregressive model은 generation speed, non-autoregressive model은 temporal modeling의 한계가 있음PALLEAutoregressive의 explicit temporal modeling과 non-autoregressive의 parallel genertion을 combine 한 pseudo-autoregressive approach를 도입Two-stage framework를 기반으로 first stage에서는 ..
[Paper 리뷰] FELLE: Autoregressive Speech Synthesis with Token-wise Coarse-to-Fine Flow Matching
FELLE: Autoregressive Speech Synthesis with Token-wise Coarse-to-Fine Flow MatchingLanguage modeling과 flow matching을 integrate 할 수 있음FELLELanguage model의 autoregressive nature와 flow matching의 generative efficacy를 기반으로 continuous-valued token을 predict추가적으로 coarse-to-fine flow matching mechanism을 통해 speech quality를 향상논문 (MM 2025) : Paper Link1. IntroductionVALL-E, VALL-E2와 같은 Large Language Model ..
[Paper 리뷰] Differentiable Reward Optimization for LLM based TTS System
Differentiable Reward Optimization for LLM based TTS SystemNeural codec language model-based Text-to-Speech system의 성능을 개선할 수 있음DiffRONeural codec token을 기반으로 reward를 directly compute 하고 Gumbel-Softmax를 사용하여 reward function을 differentiable 하도록 구성추가적으로 Multi-Task Reward model을 도입하여 다양한 perspective에서 feedback을 제공논문 (INTERSPEECH 2025) : Paper Link1. IntroductionNeural codec token Language Modeling ..