Variational autoencoder with disentanglement priors for low-resource task-specific natural language generation

Zhuang Li, Lizhen Qu*, Qiongkai Xu, Tongtong Wu, Tianyang Zhan, Gholamreza Haffari

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

1 Citation (Scopus)

Abstract

In this paper, we propose a variational autoencoder with disentanglement priors, VAE-DPRIOR, for task-specific natural language generation with none or a handful of task-specific labeled examples. In order to tackle compositional generalization across tasks, our model performs disentangled representation learning by introducing a conditional prior for the latent content space and another conditional prior for the latent label space. Both types of priors satisfy a novel property called ϵ-disentangled. We show both empirically and theoretically that the novel priors can disentangle representations even without specific regularizations as in the prior work. The content prior enables directly sampling diverse content representations from the content space learned from the seen tasks, and fuse them with the representations of novel tasks for generating semantically diverse texts in the low-resource settings. Our extensive experiments demonstrate the superior performance of our model over competitive baselines in terms of i) data augmentation in continuous zero/few-shot learning, and ii) text style transfer in the few-shot setting. The code is available at https://github.com/zhuang-li/VAE-DPrior.

Original languageEnglish
Title of host publicationProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
EditorsYoav Goldberg, Zornitsa Kozareva, Yue Zhang
Place of PublicationKerrville, TX
PublisherAssociation for Computational Linguistics
Pages10335-10356
Number of pages22
DOIs
Publication statusPublished - Dec 2022
Externally publishedYes
Event2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022 - Abu Dhabi, United Arab Emirates
Duration: 7 Dec 202211 Dec 2022

Conference

Conference2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Period7/12/2211/12/22

Fingerprint

Dive into the research topics of 'Variational autoencoder with disentanglement priors for low-resource task-specific natural language generation'. Together they form a unique fingerprint.

Cite this