[Paper 리뷰] PALLE: Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesis
PALLE: Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech SynthesisZero-Shot Text-to-Speech에서 autoregressive model은 generation speed, non-autoregressive model은 temporal modeling의 한계가 있음PALLEAutoregressive의 explicit temporal modeling과 non-autoregressive의 parallel genertion을 combine 한 pseudo-autoregressive approach를 도입Two-stage framework를 기반으로 first stage에서는 ..
[Paper 리뷰] FELLE: Autoregressive Speech Synthesis with Token-wise Coarse-to-Fine Flow Matching
FELLE: Autoregressive Speech Synthesis with Token-wise Coarse-to-Fine Flow MatchingLanguage modeling과 flow matching을 integrate 할 수 있음FELLELanguage model의 autoregressive nature와 flow matching의 generative efficacy를 기반으로 continuous-valued token을 predict추가적으로 coarse-to-fine flow matching mechanism을 통해 speech quality를 향상논문 (MM 2025) : Paper Link1. IntroductionVALL-E, VALL-E2와 같은 Large Language Model ..
[Paper 리뷰] Differentiable Reward Optimization for LLM based TTS System
Differentiable Reward Optimization for LLM based TTS SystemNeural codec language model-based Text-to-Speech system의 성능을 개선할 수 있음DiffRONeural codec token을 기반으로 reward를 directly compute 하고 Gumbel-Softmax를 사용하여 reward function을 differentiable 하도록 구성추가적으로 Multi-Task Reward model을 도입하여 다양한 perspective에서 feedback을 제공논문 (INTERSPEECH 2025) : Paper Link1. IntroductionNeural codec token Language Modeling ..
[Paper 리뷰] VALL-E2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
VALL-E2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers기존의 VALL-E를 추가적으로 개선할 수 있음VALL-E2Repetition Aware Sampling을 통해 기존 nucleus sampling process를 향상Grouped Code Modeling을 통해 inference speed와 long sequence modeling을 개선논문 (Microsoft 2025) : Paper Link1. IntroductionText-to-Speech (TTS)는 text input으로부터 high clarity, intelligibility를 가진 high-quality speech를 생성하는..
[Paper 리뷰] CosyVoice3: Towards In-the-Wild Speech Generation via Scaling-up and Post-Training
CosyVoice3: Towards In-the-Wild Speech Generation via Scaling-up and Post-Training앞선 CosyVoice2는 language coverage, domain diversity, data volume 측면에서 한계가 있음CosyVoice3Supervised multi-task training에 기반한 speech tokenizer를 도입Differentiable reward model을 위한 post-training을 적용Data size, model size scaling을 통해 다양한 domain과 text format을 지원논문 (Alibaba 2025) : Paper Link1. IntroductionZero-shot Text-to-Sp..
[Paper 리뷰] CosyVoice2: Scalable Streaming Speech Synthesis with Large Language Models
CosyVoice2: Scalable Streaming Speech Synthesis with Large Language Models기존 CosyVoice를 추가적으로 개선할 수 있음CosyVoice2Speech token의 codebook utilization을 향상하는 finite-scalar quantization을 도입Pre-trained large language model을 backbone으로 사용할 수 있도록 architecture를 streamline 하고 chunk-aware causal flow matching model을 통해 streaming/non-streaming synthesis를 지원논문 (Alibaba 2024) : Paper Link1. IntroductionZero-sh..