[Paper 리뷰] ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph ReadingText-to-Speech는 sentence에서는 우수한 성능을 보이고 있지만, paragraph/long-form reading에서는 어려움이 있음ContextSpeechGlobal text와 speech context를 sentence encoding에 incorporate 하는 memory-cached recurrence mechanism을 도입Hierarchically-structured textual semantics를 구성하여 global context enhancement의 scope를 향상추가적으로 linearized self-attention을 채택해 e..
[Paper 리뷰] SALTTS: Leveraging Self-Supervised Speech Representations for Improved Text-to-Speech Synthesis
SALTTS: Leveraging Self-Supervised Speech Representations for Improved Text-to-Speech SynthesisText-to-Speech에서 richer representation을 반영하기 위해 Self-Supervised Learning model을 활용할 수 있음SALTTSSelf-Supervised Learning representation을 reconstruct 하기 위해 encoder layer를 통해 FastSpeech2 encoder의 length-regulated output을 전달함SALTTS-parallel에서 해당 encoder representation은 auxiliary reconstruction loss로 사용되고, S..
[Paper 리뷰] Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder for High Fidelity Flow-based Speech Synthesis
Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder for High Fidelity Flow-based Speech SynthesisText-to-Speech 모델은 주로 mel-spectrogram과 같은 low-resolution intermediate representation에 의존하므로 vocoder와 acoustic model 간의 mismatch가 존재함Glow-WaveGANPre-designed intermediate representation에 의존하지 않고 GAN과 결합된 VAE를 사용하여 speech에서 latent representation을 직접 학습이후 flow-based aco..
[Paper 리뷰] GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis
GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech SynthesisMulti-speaker text-to-speech 모델을 fine-tuning 하여 limited training data로 다양한 speaker의 음성을 합성할 수 있음- BUT, 여전히 real speech sample과 비교하여 합성 결과의 품질이 떨어짐GANSpeechNon-autoregressive Text-to-Speech 모델에 adversarial training을 적용추가적으로 adversarial training에서 사용되는 feature matching loss에 대한 automatic scaling method를 도입논문 (INTERSPEE..
[Paper 리뷰] DeviceTTS: A Small-Footprint, Fast, Stable Network for On-device Text-to-Speech
DeviceTTS: A Small-Footprint, Fast, Stable Network for On-device Text-to-Speech기존의 tex-to-speech 모델은 크고 복잡한 network로 구성되기 때문에, 원활한 배포를 지원할 수 있는 on-device text-to-speech에 적합한 모델이 필요함DeviceTTSDuration predictor를 통해 encoder, decoder 간의 bridge를 제공모델 size를 줄이기 위해 Deep Feedforward Sequential Memory Network (DFSMN)을 도입추가적으로 추론 속도를 높이기 위해, mix-resolution decoder를 채택논문 (ICASSP 2021) : Paper Link1. Introd..
[Paper 리뷰] EdiTTS: Score-based Editing for Controllable Text-to-Speech
EdiTTS: Score-based Editing for Controllable Text-to-SpeechText-to-Speech를 위해 score-based modeling을 기반으로 speech editing method를 적용할 수 있음EdiTTSAdditional training이나 task-specific optimization 없이 content, pitch 측면에서 audio에 대한 targeted, granular editing을 허용Gaussian prior space에서 coarse, deliberate perturbation을 적용하여 diffusion model에서 desired behavior를 유도하고, mask와 softening kernel을 통해 target region에..