[Paper 리뷰] CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptions
CrisperWhisper: Accurate Timestamps on Verbatim Speech TranscriptionsWhisper의 tokenizer를 adjust 하여 word-level timestamps precision을 향상할 수 있음CrisperWhisperWhisper decoder의 cross-attention score에 dynamic time warping을 적용추가적인 fine-tuning을 통해 robustness를 향상논문 (INTERSPEECH 2024) : Paper Link1. IntroductionAutomatic Speech Recognition (ASR)에서 large-scale, weakly supervised learning은 뛰어난 성능을 보이고 있음특히 S..
[Paper 리뷰] WaveFM: A High-Fidelity and Efficient Vocoder based on Flow Matching
WaveFM: A High-Fidelity and Efficient Vocoder based on Flow MatchingFlow Matching은 diffusion model에 대한 robust training을 제공하지만 neural vocoder에 directly applying 하면 audio quality가 저하됨WaveFMStandard Gaussian prior 대신 mel-conditioned prior distribution을 채택하여 transportation cost를 minimizeRefined multi-resolution STFT loss를 결합하여 audio quality를 향상추가적으로 inference speed 향상을 위해 consistency distillation me..
[Paper 리뷰] Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesis
Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech SynthesisSpeech synthesis를 위해 autoregressive modeling을 활용할 수 있음CAMMulti-modal latent space를 가지는 Variational AutoEncoder, conditional probability distribution으로써 Gaussian Mixture Model을 활용하는 autoregressive model을 활용특히 Variational AutoEncoder의 latent space에서 continuous speech representation을 통해 training/inference pip..
[Paper 리뷰] ZSVC: Zero-Shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training
ZSVC: Zero-Shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial TrainingStyle voice conversion은 original speaker identity를 유지하면서 source speech의 speaking style을 desired style로 변환하는 것을 목표로 함ZSVCSpeech codec과 speech prompting mechanism을 포함한 latent diffusion model을 활용Speaking style, timbre를 disentangle 하기 위해 information bottleneck을 도입하고 Uncetainty Modeling Adaptive I..
[Paper 리뷰] ComplexDec: A Domain-Robust High-Fidelity Neural Audio Codec with Complex Spectrum Modeling
ComplexDec: A Domain-Robust High-Fidelity Neural Audio Codec with Complex Spectrum Modeling기존의 neural audio codec은 out-of-domain audio를 modeling 하는데 어려움이 있음ComplexDecOut-of-Domain robustness는 codec compression으로 인한 information loss로 인해 발생24kbps bitrate에서 해당 information loss를 완화하기 위해 complex spectral input/output을 활용논문 (ICASSP 2025) : Paper Link1. IntroductionDigital Signal Processing (DSP)-based..
[Paper 리뷰] NanoVoice: Efficient Speaker-Adaptive Text-to-Speech for Multiple Speakers
NanoVoice: Efficient Speaker-Adaptive Text-to-Speech for Multiple SpeakersMultiple speaker에 대한 adapter를 활용하여 personalized text-to-speech model을 구성할 수 있음NanoVoiceMultiple reference를 parallel fine-tuning 할 수 있는 batch-wise speaker adaptation을 활용추가적으로 speaker adaptation parameter를 줄이기 위해 parameter sharing을 도입하고, trainable scale matrix를 incorporate논문 (ICASSP 2025) : Paper Link1. IntroductionVALL-E, V..