AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We propose a novel and simple dualgenerator network architecture for text style transfer, which does not rely on any discriminators or parallel corpus for training

DGST: a Dual Generator Network for Text Style Transfer

EMNLP 2020, (2020)

Cited by: 0|Views24
Full Text
Bibtex
Weibo

Abstract

We propose DGST, a novel and simple DualGenerator network architecture for text Style Transfer. Our model employs two generators only, and does not rely on any discriminators or parallel corpus for training. Both quantitative and qualitative experiments on the Yelp and IMDb datasets show that our model gives competitive performance compar...More

Code:

Data:

0
Introduction
  • Attribute style transfer is a task which seeks to change a stylistic attribute of text, while preserving its attribute-independent information.
  • In contrast to some of the dominant approaches to style transfer such as CycleGAN (Zhu et al, 2017), the model does not employ any discriminators and yet can be trained without requiring any parallel corpus.
  • The authors achieve this by developing a novel sentence noisification approach called neighbourhood sampling, which can introduce noise to each input sentence dynamically.
  • The code of DGST is available at: https://xiao.ac/proj/dgst
Highlights
  • Attribute style transfer is a task which seeks to change a stylistic attribute of text, while preserving its attribute-independent information
  • We propose a novel and simple model architecture for text style transfer, which employs two generators only
  • The nosified sentences are used to train our style transferrers in the way similar to the training of denoising autoencoders (Vincent et al, 2008). Both quantitative and qualitative evaluation on the Yelp and IMDb benchmark datasets show that DGST gives competitive performance compared to several strong baselines which have more complicated model design
  • Following the standard evaluation practice, we evaluate the performance of our model on the textual style transfer task from two aspects: (1) Transfer Intensity: a style classifier is employed for quantifying the intensity of the transferred text
  • We propose a novel and simple dualgenerator network architecture for text style transfer, which does not rely on any discriminators or parallel corpus for training
  • Extensive experiments on two public datasets show that our model yields competitive performance compared to several strong baselines, despite of our simpler model architecture design
Methods
  • Suppose the authors have two non-parallel corpora X and Y with style Sx and Sy, the goal is training two transferrers, each of which can (i) transfer a sen-.
  • Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7131–7136, November 16–20, 2020.
  • C 2020 Association for Computational Linguistics text + noise L&(') = L&($) + noise Reconstruction L"($) text
Results
  • The two transferrers (f and g) are Stacked BiLSTM-based sequence-to-sequence models, i.e., both 4-layer BiLSTM for the encoder and decoder.
  • As shown in Table 2, for the Yelp dataset the model defeats all baselines models (apart from StyleTransformer (Multi-Class)) on both ref BLEU and self -BLEU.
  • As shown in Table 2, the model works remarkably well on both transfer intensity and preservation without requiring adversarial training or reinforcement learning, or external offline sentiment classifiers (as in Dai et al (2019)).
  • Ref -BLEU self -BLEU.
  • DeleteAndRetrieve (Li et al, 2018)
Conclusion
  • The authors propose a novel and simple dualgenerator network architecture for text style transfer, which does not rely on any discriminators or parallel corpus for training.
  • Extensive experiments on two public datasets show that the model yields competitive performance compared to several strong baselines, despite of the simpler model architecture design
Tables
  • Table1: Statistics of Datasets
  • Table2: Automatic evaluation results on Yelp and IMDb corpora, most of which are from <a class="ref-link" id="cDai_et+al_2019_a" href="#rDai_et+al_2019_a">Dai et al (2019</a>)
  • Table3: Example results from our model for the sentiment style transfer on the Yelp and IMDb datasets
  • Table4: Example transferred from the ablation study
  • Table5: Evaluation results for the ablation study
Download tables as Excel
Funding
  • This work is supported by the award made by the UK Engineering and Physical Sciences Research Council (Grant number: EP/P011829/1)
Reference
  • Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural machine translation. In International Conference on Learning Representations.
    Google ScholarLocate open access versionFindings
  • Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997–6007, Florence, Italy. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. 2019. Implicit deep latent variable models for text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3937–3947.
    Google ScholarLocate open access versionFindings
  • Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Exploration and evaluation. In Thirty-Second AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1587–1596. JMLR. org.
    Google ScholarLocate open access versionFindings
  • Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non-parallel text style transfer. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 424–434.
    Google ScholarLocate open access versionFindings
  • Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 201Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc’Aurelio Ranzato, and YLan Boureau. 2019. Multiple-attribute text rewriting. In International Conference on Learning Representations.
    Google ScholarLocate open access versionFindings
  • Juncen Li, Robin Jia, He He, and Percy Liang. 2018.
    Google ScholarFindings
  • Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865–1874, New Orleans, Louisiana. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Ruizhe Li, Xiao Li, Chenghua Lin, Matthew Collinson, and Rui Mao. 2019. A stable variational autoencoder for text modelling. In Proceedings of the 12th International Conference on Natural Language Generation, pages 594–599.
    Google ScholarLocate open access versionFindings
  • Xiao Li, Chenghua Lin, Ruizhe Li, Chaozheng Wang, and Frank Guerin. 2020. Latent space factorisation and manipulation via matrix subspace projection. In Proceedings of Machine Learning and Systems 2020, pages 3211–3221.
    Google ScholarLocate open access versionFindings
  • Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Tianxiao Shen, Jonas Mueller, Regina Barzilay, and Tommi Jaakkola. 2020. Educating text autoencoders: Latent representation guidance via denoising. In Proceedings of Machine Learning and Systems 2020, pages 9129–9139.
    Google ScholarLocate open access versionFindings
  • Youzhi Tian, Zhiting Hu, and Zhou Yu. 2018. Structured content preservation for unsupervised text style transfer. arXiv preprint arXiv:1810.06526.
    Findings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
    Google ScholarLocate open access versionFindings
  • Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096–1103.
    Google ScholarLocate open access versionFindings
  • Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. Mask and infill: Applying masked language model for sentiment transfer. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI19, pages 5271–5277. International Joint Conferences on Artificial Intelligence Organization.
    Google ScholarLocate open access versionFindings
  • Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, and Wenjie Li. 2018. Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 979–988, Melbourne, Australia. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg-Kirkpatrick. 2018. Unsupervised text style transfer using language models as discriminators. In Advances in Neural Information Processing Systems, pages 7287–7298.
    Google ScholarLocate open access versionFindings
  • Junbo Zhao, Yoon Kim, Kelly Zhang, Alexander Rush, and Yann LeCun. 2018. Adversarially regularized autoencoders. volume 80 of Proceedings of Machine Learning Research, pages 5902–5911, Stockholmsmassan, Stockholm Sweden. PMLR.
    Google ScholarLocate open access versionFindings
  • Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223–2232.
    Google ScholarLocate open access versionFindings
  • Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Advances in neural information processing systems, pages 6830–6841.
    Google ScholarLocate open access versionFindings
Author
Guanyi Chen
Guanyi Chen
Ruizhe Li
Ruizhe Li
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科