[Paper 리뷰] SALTTS: Leveraging Self-Supervised Speech Representations for Improved Text-to-Speech Synthesis
SALTTS: Leveraging Self-Supervised Speech Representations for Improved Text-to-Speech SynthesisText-to-Speech에서 richer representation을 반영하기 위해 Self-Supervised Learning model을 활용할 수 있음SALTTSSelf-Supervised Learning representation을 reconstruct 하기 위해 encoder layer를 통해 FastSpeech2 encoder의 length-regulated output을 전달함SALTTS-parallel에서 해당 encoder representation은 auxiliary reconstruction loss로 사용되고, S..
[Paper 리뷰] ItoWave: Ito Stochastic Differential Equation is All You Need for Wave Generation
ItoWave: Ito Stochastic Differential Equation is All You Need for Wave GenerationForward/reverse-time linear stochastic differential equation의 pair에 기반한 vocoder를 구성할 수 있음ItoWaveWaveform을 tractable distribution으로 변환하는 process와 해당 tractable signal을 target wave로 변환하는 두 가지의 stochastic process를 활용Original mel-spectrogram의 conditional input 하에서 meaningful audio를 생성하기 위해 noise signal에서 excess signal을 점..
[Paper 리뷰] Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech Generation
Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech GenerationMulti-codebook speech codec은 multi-sequence prediction으로 인해 efficiency와 robustness에 bottleneck이 발생함Single-CodecDisentangled VQVAE를 통해 speech를 time-invariant embedding과 phonetically-rich discrete sequence로 decouple 하는 single-codebook, single-sequence codec특히 encoder에서Temporal information을 반영하는 BLSTM module을 통해 co..
[Paper 리뷰] Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language PromptSinging Voice Synthesis는 여전히 style attribute를 explicitly control 하는 것이 어려움Prompt-SingerGender, vocal range, volume 등을 natural language prompt로 control하는 singing voice synthesis 모델Multi-scale hierarchy를 가지는 decoder-only transformer를 기반으로 melodic accuracy를 유지하면서 text-conditioned vocal range control이 가능한 range-melody de..
[Paper 리뷰] ScoreDec: A Phase-Preserving High-Fidelity Audio Codec with a Generalized Score-based Diffusion Post-Filter
ScoreDec: A Phase-Preserving High-Fidelity Audio Codec with a Generalized Score-based Diffusion Post-FilterWaveform-domain end-to-end neural codec은 low-bitrate의 coding이 가능하지만 여전히 natural audio와의 품질 차이가 존재함해당 neural codec의 성능을 향상하기 위해서는 GAN training이 필요하지만, original phase information preserving을 방해한다는 문제가 있음ScoreDecGAN training에서 original phase preserving을 위해, complex spectral domain에서 score-base..
[Paper 리뷰] Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder for High Fidelity Flow-based Speech Synthesis
Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder for High Fidelity Flow-based Speech SynthesisText-to-Speech 모델은 주로 mel-spectrogram과 같은 low-resolution intermediate representation에 의존하므로 vocoder와 acoustic model 간의 mismatch가 존재함Glow-WaveGANPre-designed intermediate representation에 의존하지 않고 GAN과 결합된 VAE를 사용하여 speech에서 latent representation을 직접 학습이후 flow-based aco..