[Paper 리뷰] Variable Bitrate Residual Vector Quantization for Audio Coding
Variable Bitrate Residual Vector Quantization for Audio CodingNeural audio codec은 rate-distortion trade-off 측면에서 suboptimal 함VRVQFrame 당 사용되는 codebook 수를 adapting 하여 efficient coding을 지원Importance map을 binary importance mask로 transform 하는 non-differentiable masking operation에 대한 gradient estimation method를 도입논문 (ICASSP 2025) : Paper Link1. Introduction최근 SoundStream, EnCodec, DAC와 같은 Residual Ve..
[Paper 리뷰] PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning
PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec LearningNeural speech codec은 Residual Vector Quantization으로 인한 reconstruction의 한계가 있음PURE CodecPre-trained speech enhancement model을 활용하여 multi-stage quantization을 guidingFirst stage에서는 low-entropy, denoised speech embedding을 reconstruct 하고 second stage에서는 residual high-entropy component를 encode논문 (ASRU 2025) : Paper Link1. I..
[Paper 리뷰] Language-Codec: Bridging Discrete Codec Representations and Speech Language Models
Language-Codec: Bridging Discrete Codec Representations and Speech Language ModelsDiscrete acoustic codec은 speech language model에서 intermediate representation으로 사용됨Language-CodecMasked Channel Residual Vector Quantization을 도입하여 initial codebook의 excessive information 문제를 해결추가적으로 Fourier transform structure, attention block, refined discriminator를 적용논문 (ACL 2025) : Paper Link1. IntroductionVALL-E..
[Paper 리뷰] SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound대부분의 neural codec은 high bitrate에서 동작하고 narrow domain을 가짐SemantiCodecSpeech, general sound, music 등의 다양한 domain을 100 token/sec 이하의 token으로 compress$k$-means clustering을 통해 discretize 된 Self-Supervised Pre-Trained Audio Masked AutoEncoder와 acoustic encoder로 구성된 dual-encoder architecture를 활용논문 (JSTSP 2024) : Paper Link1. Intro..
[Paper 리뷰] TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling
TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling기존의 speech tokenizer는 high frame rate와 auxiliary pre-trained model에 대한 의존성, complex training process와 같은 한계점이 존재함TaDiCodecDiffusion AutoEncoder를 활용해 quantization, reconstruction에 대한 end-to-end optimization을 수행Text guidance를 diffusion decoder에 integrate 하여 optimal compression을 달성논문 (NeurIPS 2025) : Paper Link1. Introduct..
[Paper 리뷰] FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks
FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks기존의 neural codec은 high bitrate, semantic/acoustic information loss의 문제가 있음FocalCodecFocal modulation을 기반으로 single binary codebook을 사용하여 speech를 compressSemantic/acoustic information을 preserve 하여 다양한 downstream task에서 우수한 성능을 달성논문 (NeurIPS 2025) : Paper Link1. IntroductionAudioLM, AudioGen과 같은 speech language model은 token-based sp..