[Paper 리뷰] StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow MatchingZero-Shot Voice Conversion은 다음의 한계점이 있음- Style과 timbre를 서로 다른 unseen speaker에게 independently transfer 할 수 없음- Autoregressive modeling이나 sampling step으로 인해 추론 속도가 느림- Converted sample의 품질과 similarity는 여전히 만족스럽지 않음StableVCSpeech를 linguistic content, timbre, style로 decompose하고 conditional flow matching module을 사용하..
[Paper 리뷰] VoiceMixer: Adversarial Voice Style Mixup
VoiceMixer: Adversarial Voice Style MixupVoice conversion은 source speech와 voice style을 충분히 decompose 하지 못해 여전히 한계가 있음VoiceMixerSelf-supervised representation learning을 활용한 information bottleneck을 통해 content와 style을 decompose 함각 information에 대한 adversarial feedback을 통해 더 나은 generalization을 달성논문 (NeurIPS 2021) : Paper Link1. IntroductionVoice Conversion (VC)는 source speaker의 content information은 유..
[Paper 리뷰] Expressive-VC: Highly Expressive Voice Conversion with Attention Fusion of Bottleneck and Perturbation Features
Expressive-VC: Highly Expressive Voice Conversion with Attention Fusion of Bottleneck and Perturbation FeaturesVoice conversion은 speaker similarity, intelligibility, expressiveness 측면에서 한계가 있음Expressive-VCNeural bottleneck feature approach와 information perturbation approach를 결합한 end-to-end voice conversion modelBottleneck feature encoder와 perturbe wav encoder를 사용하여 linguistic, para-linguistic fe..
[Paper 리뷰] DualVC3: Leveraging Language Model Generated Pseudo Context for End-to-End Low Latency Streaming Voice Conversion
DualVC3: Leveraging Language Model Generated Pseudo Context for End-to-End Low Latency Streaming Voice Conversion최근의 DualVC2는 180ms의 latency로 streaming voice conversion이 가능함- BUT, recognition-synthesis framework로 인해 end-to-end optimization이 어렵고 short chunk를 사용하는 경우 instability가 증가함DualVC3Speaker-independent semantic token을 사용하여 content encoder training을 guideLanguage model을 content encoder outpu..
[Paper 리뷰] Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-Shot Speaker Adaptation
Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-Shot Speaker AdaptationVoice Conversion은 여전히 inaccurate pitch와 low speaker adaptation 문제를 가지고 있음Diff-HierVC2가지 diffusion model을 기반으로 하는 hierarchical voice conversion model- Target voice style로 $F_{0}$를 효과적으로 생성할 수 있는 DiffPitch를 도입하고,- 이후 생성된 $F_{0}$를 DiffVoice에 전달하여 target voice styl..
[Paper 리뷰] StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversion
StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice ConversionLanguage Model을 활용하여 zero-shot voice conversion 성능을 향상할 수 있음- BUT, 기존 방식은 offline conversion으로 인해 complete source speech 만을 요구하므로 real-time application에서 활용하기 어려움StreamVoiceStreaming capability를 위해 temporal independent acoustic predictor를 포함한 fully causal context-aware Language Model을 도입- 이를 통해 comple..