[Paper 리뷰] VibeVoice: Expressive Podcast Generation with Next-Token Diffusion
VibeVoice: Expressive Podcast Generation with Next-Token DiffusionPodcast와 같은 long-form, multi-speaker conversational audio를 생성하기 위해서는 Text-to-Speech system에서 scalability, speaker consistency, natural turn-taking를 보장할 수 있어야 함VibeVoice7.5 ultra-low frame rate의 continuous speech tokenizer를 활용해 long sequence efficiency를 개선추가적으로 next-token diffusion framework를 통해 expressive podcast generation을 지원논문 ..
[Paper 리뷰] VoxCPM: Hierarchical Semantic-Acoustic Modeling via Semi-Discrete Residual Representations for Expressive End-to-End Speech Synthesis
VoxCPM: Hierarchical Semantic-Acoustic Modeling via Semi-Discrete Residual Representations for Expressive End-to-End Speech SynthesisSpeech tokenizer 기반의 multi-stage speech synthesis는 semantic-acoustic divide로 인한 trade-off가 존재함VoxCPMSemi-discrete residual representation 기반의 hierarchical semantic-acoustic modeling을 적용추가적으로 natural specialization을 위한 differentiable quantization bottleneck을 도입논문 (I..
[Paper 리뷰] KALL-E: Autoregressive Speech Synthesis with Next-Distribution Prediction
KALL-E: Autoregressive Speech Synthesis with Next-Distribution PredictionText-to-Speech를 위해 autoregressive language model을 활용할 수 있음KALL-EFlow-VAE를 활용하여 waveform으로부터 continuous latent speech representation을 추출Single AR Transformer를 통해 text로부터 해당 continuous speech distribution을 predict논문 (AAAI 2026) : Paper Link1. IntroductionVALL-E와 같이 Text-to-Speech (TTS)를 위해 Large Language Model (LLM)을 활용할 수 있음..
[Paper 리뷰] RRPO: Robust Reward Policy Optimization for LLM-based Emotional TTS
RRPO: Robust Reward Policy Optimization for LLM-based Emotional TTSEmotion contorl과 같은 nuanced task에서 기존의 reward optimization method는 reward hacking 문제가 발생함RRPOHybrid regularization을 활용하여 reward signal이 reliably align 되도록 유도특히 policy가 detrimental shortcut을 abandon 하고 emotion의 complex feature를 학습하도록 함논문 (ICASSP 2026) : Paper Link1. IntroductionCosyVoice2와 같이 Large Language Model (LLM)을 활용하면 우수한 T..
[Paper 리뷰] Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
Efficient Speech Language Modeling via Energy Distance in Continuous Latent SpaceSpeech language model은 discretization으로 인한 한계가 있음SLEDSpeech waveform을 continuous latent representation의 sequence로 encodingEnergy distance objective를 사용하여 autoregressive modeling을 수행논문 (NeurIPS 2025) : Paper Link1. IntroductionSpeech audio는 integer/floating-point range내의 value를 가지는 lengthy sampling point sequence로 re..
[Paper 리뷰] Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance
Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free GuidanceAutoregressive speech token generation model은 hallucination과 undesired vocalization의 문제가 있음Koel-TTSPreference alignment와 Classifier Free Guidance를 활용하여 Language Model의 contextual adherence를 향상특히 speech recognition model에서 derive 된 automatic metric을 사용하여 model output을 rank 하고 conditional, uncondi..