[Paper 리뷰] RFWave: Multi-Band Rectified Flow for Audio Waveform Reconstruction
RFWave: Multi-Band Rectified Flow for Audio Waveform ReconstructionDiffusion model은 waveform reconstruction에 효과적이지만 상당한 sampling step이 필요하므로 latency 문제가 존재함RFWaveComplex spectrogram을 생성하고 frame-level에서 모든 subband를 simultaneously process 함Straight transport trajectory를 위해 Rectified Flow를 도입논문 (ICLR 2025) : Paper Link1. IntroductionAudio waveform reconstruction은 raw audio data에서 derive 된 low-dimen..
[Paper 리뷰] PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation
PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform GenerationHigh-resolution waveform signal의 natural periodic feature를 explicitly disentangle 할 수 있는 generator가 필요함PeriodWaveVector field를 추정할 때 waveform signal의 periodic feature를 capture 하는 period-aware flow matching estimator를 도입Waveform signal의 periodic feature를 capture 하는 multi-period estimator를 활용추가적으로 waveform generation에서 hig..
[Paper 리뷰] DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors
DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific FactorsLarge-scale Latent Diffusion Model은 various modality에 대해 우수한 content generation 성능을 보여주고 있지만, text-to-speech에서는 phoneme, duration에 의존해야 하므로 scalability의 한계가 있음DiTTo-TTSDomain-specific factor를 제거한 Latent Diffusion Model 기반의 text-to-speech model기존 U-Net 대신 Diffusion Transformer를 채택하고 speech length predicto..
[Paper 리뷰] UniAudio: Towards Universal Audio Generation with Large Language Models
UniAudio: Towards Universal Audio Generation with Large Language Models다양한 task를 unified manner로 처리할 수 있는 universal audio generation model이 필요함UniAudioLarge Language Model-based audio generation model을 구성해 phoneme, text description, audio 등의 다양한 input condition을 기반으로 speech, sound, music, singing voice 등을 생성Model performance와 efficiency를 향상하기 위한 audio tokenization과 language model architecture를 설..
[Paper 리뷰] Whisper: Robust Speech Recognition via Large-Scale Weak Supervision
Whisper: Robust Speech Recognition via Large-Scale Weak SupervisionAudio transcripting을 위해 large data로 speech processing system을 training 할 수 있음Whisper680,000 hours의 multilingual, multitask supervision으로 model을 scalingDataset에 대한 specific fine-tuning 없이도 뛰어난 accuracy, robustness를 달성논문 (ICML 2023) : Paper Link1. IntroductionSpeech recognition은 wav2vec 2.0의 unsupervised pre-training technique을 주로 ..