On the Semantic Latent Space of Diffusion-Based Text-to-Speech Models
CoRR(2024)
摘要
The incorporation of Denoising Diffusion Models (DDMs) in the Text-to-Speech
(TTS) domain is rising, providing great value in synthesizing high quality
speech. Although they exhibit impressive audio quality, the extent of their
semantic capabilities is unknown, and controlling their synthesized speech's
vocal properties remains a challenge. Inspired by recent advances in image
synthesis, we explore the latent space of frozen TTS models, which is composed
of the latent bottleneck activations of the DDM's denoiser. We identify that
this space contains rich semantic information, and outline several novel
methods for finding semantic directions within it, both supervised and
unsupervised. We then demonstrate how these enable off-the-shelf audio editing,
without any further training, architectural changes or data requirements. We
present evidence of the semantic and acoustic qualities of the edited audio,
and provide supplemental samples:
https://latent-analysis-grad-tts.github.io/speech-samples/.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要