[Paper 리뷰] FunCodec: A Fundamental, Reproducible and Integrable Open-Source Toolkit for Neural Speech Codec
FunCodec: A Fundamental, Reproducible and Integrable Open-Source Toolkit for Neural Speech CodecSoundStream, EnCodec과 같은 neural codec에 대한 open-source toolkit이 필요함FunCodecDownstream task에 easily integrate 될 수 있는 open-source codecLower computation, parameter complexity를 가지는 frequency-domain codec을 지원논문 (ICASSP 2024) : Paper Link1. IntroductionSpeech codec은 speech를 compact representation으로 encode 하..
[Paper 리뷰] ComplexDec: A Domain-Robust High-Fidelity Neural Audio Codec with Complex Spectrum Modeling
ComplexDec: A Domain-Robust High-Fidelity Neural Audio Codec with Complex Spectrum Modeling기존의 neural audio codec은 out-of-domain audio를 modeling 하는데 어려움이 있음ComplexDecOut-of-Domain robustness는 codec compression으로 인한 information loss로 인해 발생24kbps bitrate에서 해당 information loss를 완화하기 위해 complex spectral input/output을 활용논문 (ICASSP 2025) : Paper Link1. IntroductionDigital Signal Processing (DSP)-based..
[Paper 리뷰] RepCodec: A Speech Representation Codec for Speech Tokenization
RepCodec: A Speech Representation Codec for Speech TokenizationDiscrete speech tokenization은 large language model에서 유용하게 활용되지만 discretization으로 인해 information loss가 발생함RepCodecSpeech encoder에서 speech representation을 reconstruction 하여 vector quantization codebook을 학습Speech encoder, Codec encoder, Vector quantization codebook으로 구성된 pipeline을 통해 speech waveform을 semantic token으로 변환논문 (ACL 2024) : P..
[Paper 리뷰] Generative De-quantization for Neural Speech Codec via Latent Diffusion
Generative De-quantization for Neural Speech Codec via Latent DiffusionLow-bitrate speech coding에서 end-to-end network는 compact, expressive feature와 powerful decoder를 학습하는 것을 목표로 함- BUT, 여전히 complexity와 speech quality 측면에서 한계가 있음LaDiffCodecLow-dimensional discrete token을 학습하기 위해 end-to-end codec을 구성Latent diffusion model을 사용하여 coded feature를 high-dimensional continuous space로 de-quantize추가적으로 ove..
[Paper 리뷰] Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech Generation
Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech GenerationMulti-codebook speech codec은 multi-sequence prediction으로 인해 efficiency와 robustness에 bottleneck이 발생함Single-CodecDisentangled VQVAE를 통해 speech를 time-invariant embedding과 phonetically-rich discrete sequence로 decouple 하는 single-codebook, single-sequence codec특히 encoder에서Temporal information을 반영하는 BLSTM module을 통해 co..
[Paper 리뷰] ScoreDec: A Phase-Preserving High-Fidelity Audio Codec with a Generalized Score-based Diffusion Post-Filter
ScoreDec: A Phase-Preserving High-Fidelity Audio Codec with a Generalized Score-based Diffusion Post-FilterWaveform-domain end-to-end neural codec은 low-bitrate의 coding이 가능하지만 여전히 natural audio와의 품질 차이가 존재함해당 neural codec의 성능을 향상하기 위해서는 GAN training이 필요하지만, original phase information preserving을 방해한다는 문제가 있음ScoreDecGAN training에서 original phase preserving을 위해, complex spectral domain에서 score-base..