[Paper 리뷰] VALL-E2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers
VALL-E2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers기존의 VALL-E를 추가적으로 개선할 수 있음VALL-E2Repetition Aware Sampling을 통해 기존 nucleus sampling process를 향상Grouped Code Modeling을 통해 inference speed와 long sequence modeling을 개선논문 (Microsoft 2025) : Paper Link1. IntroductionText-to-Speech (TTS)는 text input으로부터 high clarity, intelligibility를 가진 high-quality speech를 생성하는..
[Paper 리뷰] CosyVoice3: Towards In-the-Wild Speech Generation via Scaling-up and Post-Training
CosyVoice3: Towards In-the-Wild Speech Generation via Scaling-up and Post-Training앞선 CosyVoice2는 language coverage, domain diversity, data volume 측면에서 한계가 있음CosyVoice3Supervised multi-task training에 기반한 speech tokenizer를 도입Differentiable reward model을 위한 post-training을 적용Data size, model size scaling을 통해 다양한 domain과 text format을 지원논문 (Alibaba 2025) : Paper Link1. IntroductionZero-shot Text-to-Sp..
[Paper 리뷰] CosyVoice2: Scalable Streaming Speech Synthesis with Large Language Models
CosyVoice2: Scalable Streaming Speech Synthesis with Large Language Models기존 CosyVoice를 추가적으로 개선할 수 있음CosyVoice2Speech token의 codebook utilization을 향상하는 finite-scalar quantization을 도입Pre-trained large language model을 backbone으로 사용할 수 있도록 architecture를 streamline 하고 chunk-aware causal flow matching model을 통해 streaming/non-streaming synthesis를 지원논문 (Alibaba 2024) : Paper Link1. IntroductionZero-sh..
[Paper 리뷰] MELLE: Autoregressive Speech Synthesis without Vector Quantization
MELLE: Autoregressive Speech Synthesis without Vector QuantizationText-to-Speech를 위해 continuous-valued token based language modeling을 활용할 수 있음MELLESpectrogram Flux loss를 사용하여 continuous-valued token distribution을 modelingVariational inference를 incorporate 하여 diversity, robustness를 향상논문 (ACL 2025) : Paper Link1. IntroductionNext-token prediction은 previous token을 condition으로 하여 next discrete token..
[Paper 리뷰] DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation
DiTAR: Diffusion Transformer Autoregressive Modeling for Speech GenerationDiffusion model과 autoregressive model을 결합하면 computational load와 suboptimal outcome이 발생함DiTARPatch generation을 위해 divide-and-conquer strategy를 도입Langauge model은 aggregated patch embedding을 처리한 다음, diffusion Transformer를 통해 next patch를 subsequently generate추론 시에는 reverse diffusion ODE 중 noise introducing time point를 temperat..
[Paper 리뷰] ELLA-V: Stable Neural Codec Language Modeling with Alignment-Guided Sequence Recording
ELLA-V: Stable Neural Codec Language Modeling with Alignment-Guided Sequence RecordingAcoustic, linguistic prompt에 기반한 language model은 zero-shot audio synthesis에서 우수한 성능을 보임ELLA-VPhoneme level에서 synthesized audio에 대한 fine-grained control을 지원Acoustic token ahead에 phoneme token이 appear 할 때 acoustic, phoneme token sequence를 interleaving논문 (AAAI 2025) : Paper Link1. IntroductionZero-shot Text-to-Spe..