[Paper 리뷰] ComplexDec: A Domain-Robust High-Fidelity Neural Audio Codec with Complex Spectrum Modeling
ComplexDec: A Domain-Robust High-Fidelity Neural Audio Codec with Complex Spectrum Modeling기존의 neural audio codec은 out-of-domain audio를 modeling 하는데 어려움이 있음ComplexDecOut-of-Domain robustness는 codec compression으로 인한 information loss로 인해 발생24kbps bitrate에서 해당 information loss를 완화하기 위해 complex spectral input/output을 활용논문 (ICASSP 2025) : Paper Link1. IntroductionDigital Signal Processing (DSP)-based..
[Paper 리뷰] RepCodec: A Speech Representation Codec for Speech Tokenization
RepCodec: A Speech Representation Codec for Speech TokenizationDiscrete speech tokenization은 large language model에서 유용하게 활용되지만 discretization으로 인해 information loss가 발생함RepCodecSpeech encoder에서 speech representation을 reconstruction 하여 vector quantization codebook을 학습Speech encoder, Codec encoder, Vector quantization codebook으로 구성된 pipeline을 통해 speech waveform을 semantic token으로 변환논문 (ACL 2024) : P..
[Paper 리뷰] Generative De-quantization for Neural Speech Codec via Latent Diffusion
Generative De-quantization for Neural Speech Codec via Latent DiffusionLow-bitrate speech coding에서 end-to-end network는 compact, expressive feature와 powerful decoder를 학습하는 것을 목표로 함- BUT, 여전히 complexity와 speech quality 측면에서 한계가 있음LaDiffCodecLow-dimensional discrete token을 학습하기 위해 end-to-end codec을 구성Latent diffusion model을 사용하여 coded feature를 high-dimensional continuous space로 de-quantize추가적으로 ove..
[Paper 리뷰] Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech Generation
Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech GenerationMulti-codebook speech codec은 multi-sequence prediction으로 인해 efficiency와 robustness에 bottleneck이 발생함Single-CodecDisentangled VQVAE를 통해 speech를 time-invariant embedding과 phonetically-rich discrete sequence로 decouple 하는 single-codebook, single-sequence codec특히 encoder에서Temporal information을 반영하는 BLSTM module을 통해 co..
[Paper 리뷰] ScoreDec: A Phase-Preserving High-Fidelity Audio Codec with a Generalized Score-based Diffusion Post-Filter
ScoreDec: A Phase-Preserving High-Fidelity Audio Codec with a Generalized Score-based Diffusion Post-FilterWaveform-domain end-to-end neural codec은 low-bitrate의 coding이 가능하지만 여전히 natural audio와의 품질 차이가 존재함해당 neural codec의 성능을 향상하기 위해서는 GAN training이 필요하지만, original phase information preserving을 방해한다는 문제가 있음ScoreDecGAN training에서 original phase preserving을 위해, complex spectral domain에서 score-base..
[Paper 리뷰] Fewer-Token Neural Speech Codec with Time-Invariant Codes
Fewer-Token Neural Speech Codec with Time-Invariant CodesNeural codec은 speech를 discrete token으로 변환하는 데 사용되지만, excessive token sequence는 오히려 prediction accuracy에 부정적인 영향을 줄 수 있음TiCodecTime-invariant information을 별도의 code로 encoding/quantizing하여 encoding에 사용되는 frame-level information의 양을 줄임Utterance에서 time-invariant code의 consistency를 향상하기 위해, time-invariant encoding consistency loss를 도입논문 (ICASSP ..