[Paper 리뷰] SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound대부분의 neural codec은 high bitrate에서 동작하고 narrow domain을 가짐SemantiCodecSpeech, general sound, music 등의 다양한 domain을 100 token/sec 이하의 token으로 compress$k$-means clustering을 통해 discretize 된 Self-Supervised Pre-Trained Audio Masked AutoEncoder와 acoustic encoder로 구성된 dual-encoder architecture를 활용논문 (JSTSP 2024) : Paper Link1. Intro..
[Paper 리뷰] Metis: A Foundation Speech Generation Model with Masked Generative Pre-training
Metis: A Foundation Speech Generation Model with Masked Generative Pre-trainingMasked Generative Modeling을 활용하여 다양한 speech generation task에 fine-tuning 되는 speech foundation model을 구성할 수 있음MetisSelf-Supervised Learning token과 acoustic token에 대한 2가지 discrete speech representation을 활용Additional condition 없이 300K hours의 speech data에 대해 masked generative pre-training을 수행논문 (NeurIPS 2025) : Paper Link..
[Paper 리뷰] Word-Level Emotional Expression Control in Zero-Shot Text-to-Speech Synthesis
Word-Level Emotional Expression Control in Zero-Shot Text-to-Speech Synthesis대부분의 emotional Text-to-Speech는 word-level control이 어려움WeSConPre-trained zero-shot Text-to-Speech model로부터 emotion, speaking rate를 control 하는 self-training frameworkWord-level expressive synthesis를 guide 하기 위한 transition-smoothing strategy, dynamic speed control mechanism을 도입추론 시에는 dynamic emotional attention bias mechan..
[Paper 리뷰] TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling
TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling기존의 speech tokenizer는 high frame rate와 auxiliary pre-trained model에 대한 의존성, complex training process와 같은 한계점이 존재함TaDiCodecDiffusion AutoEncoder를 활용해 quantization, reconstruction에 대한 end-to-end optimization을 수행Text guidance를 diffusion decoder에 integrate 하여 optimal compression을 달성논문 (NeurIPS 2025) : Paper Link1. Introduct..
[Paper 리뷰] BlockDecoder: Boosting ASR Decoders with Context and Merger Modules
BlockDecoder: Boosting ASR Decoders with Context and Merger ModulesAttention-based Encoder-Decoder model에서 decoder는 Automatic Speech Recognition output을 autoregressively generate 함- 특히 initial layer는 textual context를 build 하고 later layer는 acoustic, textual informaiton을 merge 함BlockDecoderPurely text-based text encoder와 information을 combine 하는 merger를 도입Encoder representation을 reuse 하고 text encod..
[Paper 리뷰] Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesis
Shallow Flow Matching for Coarse-to-Fine Text-to-Speech SynthesisFlow Matching-based Text-to-Speech model을 개선할 수 있음Shallow Flow Matching (SFM)Coarse representation으로부터 Flow Matching path를 따라 intermediate state를 construct해당 state의 temporal position을 adaptively determine 하기 위해 orthogonal projection을 도입논문 (NeurIPS 2025) : Paper Link1. IntroductionVoiceBox, ReFlow-TTS, VoiceFlow와 같은 Flow Matching (F..