[Paper 리뷰] Wav2Vec-Switch: Contrastive Learning from Original-Noisy Speech Pairs for Robust Speech Recognition
Wav2Vec-Switch: Contrastive Learning from Original-Noisy Speech Pairs for Robust Speech RecognitionSelf-Supervised Learning framework는 noise robustness를 고려하지 않음Wav2Vec-SwitchOriginal-noisy speech pair를 Wav2Vec 2.0 network에 simultaneously feedOriginal, noisy speech에 대한 quantized representation을 서로에 대한 additional prediction target으로 활용논문 (ICASSP 2022) : Paper Link1. IntroductionSpeech task에 대한 Sel..
[Paper 리뷰] CSSinger: End-to-End Chunkwise Streaming Singing Voice Synthesis System based on Conditional Variational Autoencoder
CSSinger: End-to-End Chunkwise Streaming Singing Voice Synthesis System based on Conditional Variational AutoencoderEnd-to-End modeling을 singing voice synthesis에 적용하면 우수한 합성 성능을 달성할 수 있음CSSingerEnd-to-End model의 latency 절감을 위해 Chunkwise Streaming inference를 도입Variational Autoencoder의 latent representation을 활용한 fully end-to-end streaming audio synthesis를 지원논문 (AAAI 2025) : Paper Link1. Introducti..
[Paper 리뷰] Wav2Vec-Aug: Improved Self-Supervised Training with Limited Data
Wav2Vec-Aug: Improved Self-Supervised Training with Limited Data다양한 language에 대한 unlabeled data의 부족으로 인해 speech representation에 대한 Self-Supervised Learning은 여전히 한계가 있음Wav2Vec-AugWav2Vec 2.0 pre-training에 data augmentation을 적용Limited available data를 가지는 domain에 대해 Self-Supervised Learning을 적용논문 (INTERSPEECH 2022) : Paper Link1. IntroductionSelf-Supervised Learning (SSL)은 unlabeld speech로부터 repres..
[Paper 리뷰] TechSinger: Technique Controllable Multilingual Singing Voice Synthesis via Flow Matching
TechSinger: Technique Controllable Mulitlingual Singing Voice Synthesis via Flow MatchingSinging Voice Synthesis는 intensity, mixed voice, falsetto 등에 대한 precise control을 제공하지 않음TechSinger다양한 technique에 대한 expressive control을 지원하기 위해 flow-matching-based model을 도입Training data의 diversity를 향상하기 위해 phoneme-level technique lable로 dataset을 automatically annotate 하는 technique detection model을 활용Prompt-..
[Paper 리뷰] SECodec: Structural Entropy-based Compressive Speech Representation Codec for Speech Language Models
SECodec: Structural Entropy-based Compressive Speech Representation Codec for Speech Language ModelsLarge Language Model을 위한 기존의 speech representation discretization method는 Euclidean distance-based quantization이나 pre-defined codebook에 의존함SECodecSpeech를 graph로 modeling 하고 graph 내의 speech feature node를 clustering 한 다음, 2D Strutural Entropy를 minimize 하여 codebook을 추출- 2D SE minimization principle을 ..
[Paper 리뷰] SEVC: Voice Conversion via Structural Entropy
SEVC: Voice Conversion via Structural Entropy기존의 voice conversion method는 prosody leakage, speech representation blurring의 문제가 있음SEVCSource, reference speech에서 self-supervised representation을 추출하고 reference speech representation을 graph로 구축이후 2D Structural Entropy를 사용하여 semantically similar representation을 clustering- Voice conversion 시 source representation의 각 frame을 new node로 취급하고, SE를 통해 각 nod..
[Paper 리뷰] LiveSpeech: Low-Latency Zero-Shot Text-to-Speech via Autoregressive Modeling of Audio Discrete Codes
LiveSpeech: Low-Latency Zero-Shot Text-to-Speech via Autoregressive Modeling of Audio Discrete CodesNeural audio codec을 통해 zero-shot text-to-speech가 가능하지만 low-latency scenario에서 활용하기 어려움LiveSpeech각 frame의 codebook contribution을 고려한 adaptive codebook loss를 도입Codebook을 grouping 하고 해당 group에 대한 parallel processing을 수행논문 (INTERSPEECH 2024) : Paper Link1. IntroductionNaturalSpeech2와 같은 Zero-shot Text..
[Paper 리뷰] ZCS-CDiff: A Zero-Shot Code-Switching TTS System with Conformer-Based Diffusion Model
ZCS-CDiff: A Zero-Shot Code-Switching TTS System with Conformer-Based Diffusion ModelCode-Switching Text-to-Speech는 zero-shot scenario에서 활용하기에 한계가 있음ZCS-CDiffSpeech feature를 disentangle 하고 diffusion model을 사용하여 해당 disentangled attribute를 modelingConformer-based WaveNet을 denoising network로 활용하여 attribute modeling을 개선추가적으로 speaker-assist module을 통해 speaker similarity를 향상논문 (ICASSP 2025) : Paper Li..
[Paper 리뷰] MB-iSTFT-VITS: Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform
MB-iSTFT-VITS: Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier TransformLightweight end-to-end text-to-speech model이 필요함MB-iSTFT-VITSComputationally expensive component를 simple inverse Short-Time Fourier Transform으로 replaceFixed/trainable synthesis filter를 가지는 multi-band generation을 통해 waveform을 생성논문 (ICASSP 2023) : Paper Link1. I..