AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
We propose a novel and simple dualgenerator network architecture for text style transfer, which does not rely on any discriminators or parallel corpus for training
DGST: a Dual Generator Network for Text Style Transfer
EMNLP 2020, (2020)
We propose DGST, a novel and simple DualGenerator network architecture for text Style Transfer. Our model employs two generators only, and does not rely on any discriminators or parallel corpus for training. Both quantitative and qualitative experiments on the Yelp and IMDb datasets show that our model gives competitive performance compar...More
PPT (Upload PPT)
- Attribute style transfer is a task which seeks to change a stylistic attribute of text, while preserving its attribute-independent information.
- In contrast to some of the dominant approaches to style transfer such as CycleGAN (Zhu et al, 2017), the model does not employ any discriminators and yet can be trained without requiring any parallel corpus.
- The authors achieve this by developing a novel sentence noisification approach called neighbourhood sampling, which can introduce noise to each input sentence dynamically.
- The code of DGST is available at: https://xiao.ac/proj/dgst
- Attribute style transfer is a task which seeks to change a stylistic attribute of text, while preserving its attribute-independent information
- We propose a novel and simple model architecture for text style transfer, which employs two generators only
- The nosified sentences are used to train our style transferrers in the way similar to the training of denoising autoencoders (Vincent et al, 2008). Both quantitative and qualitative evaluation on the Yelp and IMDb benchmark datasets show that DGST gives competitive performance compared to several strong baselines which have more complicated model design
- Following the standard evaluation practice, we evaluate the performance of our model on the textual style transfer task from two aspects: (1) Transfer Intensity: a style classifier is employed for quantifying the intensity of the transferred text
- We propose a novel and simple dualgenerator network architecture for text style transfer, which does not rely on any discriminators or parallel corpus for training
- Extensive experiments on two public datasets show that our model yields competitive performance compared to several strong baselines, despite of our simpler model architecture design
- Suppose the authors have two non-parallel corpora X and Y with style Sx and Sy, the goal is training two transferrers, each of which can (i) transfer a sen-.
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7131–7136, November 16–20, 2020.
- C 2020 Association for Computational Linguistics text + noise L&(') = L&($) + noise Reconstruction L"($) text
- The two transferrers (f and g) are Stacked BiLSTM-based sequence-to-sequence models, i.e., both 4-layer BiLSTM for the encoder and decoder.
- As shown in Table 2, for the Yelp dataset the model defeats all baselines models (apart from StyleTransformer (Multi-Class)) on both ref BLEU and self -BLEU.
- As shown in Table 2, the model works remarkably well on both transfer intensity and preservation without requiring adversarial training or reinforcement learning, or external offline sentiment classifiers (as in Dai et al (2019)).
- Ref -BLEU self -BLEU.
- DeleteAndRetrieve (Li et al, 2018)
- The authors propose a novel and simple dualgenerator network architecture for text style transfer, which does not rely on any discriminators or parallel corpus for training.
- Extensive experiments on two public datasets show that the model yields competitive performance compared to several strong baselines, despite of the simpler model architecture design
- Table1: Statistics of Datasets
- Table2: Automatic evaluation results on Yelp and IMDb corpora, most of which are from <a class="ref-link" id="cDai_et+al_2019_a" href="#rDai_et+al_2019_a">Dai et al (2019</a>)
- Table3: Example results from our model for the sentiment style transfer on the Yelp and IMDb datasets
- Table4: Example transferred from the ablation study
- Table5: Evaluation results for the ablation study
- This work is supported by the award made by the UK Engineering and Physical Sciences Research Council (Grant number: EP/P011829/1)
- Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural machine translation. In International Conference on Learning Representations.
- Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997–6007, Florence, Italy. Association for Computational Linguistics.
- Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. 2019. Implicit deep latent variable models for text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3937–3947.
- Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Exploration and evaluation. In Thirty-Second AAAI Conference on Artificial Intelligence.
- Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1587–1596. JMLR. org.
- Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non-parallel text style transfer. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 424–434.
- Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 201Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computational Linguistics.
- Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc’Aurelio Ranzato, and YLan Boureau. 2019. Multiple-attribute text rewriting. In International Conference on Learning Representations.
- Juncen Li, Robin Jia, He He, and Percy Liang. 2018.
- Delete, retrieve, generate: a simple approach to sentiment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865–1874, New Orleans, Louisiana. Association for Computational Linguistics.
- Ruizhe Li, Xiao Li, Chenghua Lin, Matthew Collinson, and Rui Mao. 2019. A stable variational autoencoder for text modelling. In Proceedings of the 12th International Conference on Natural Language Generation, pages 594–599.
- Xiao Li, Chenghua Lin, Ruizhe Li, Chaozheng Wang, and Frank Guerin. 2020. Latent space factorisation and manipulation via matrix subspace projection. In Proceedings of Machine Learning and Systems 2020, pages 3211–3221.
- Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
- Tianxiao Shen, Jonas Mueller, Regina Barzilay, and Tommi Jaakkola. 2020. Educating text autoencoders: Latent representation guidance via denoising. In Proceedings of Machine Learning and Systems 2020, pages 9129–9139.
- Youzhi Tian, Zhiting Hu, and Zhou Yu. 2018. Structured content preservation for unsupervised text style transfer. arXiv preprint arXiv:1810.06526.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
- Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096–1103.
- Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. Mask and infill: Applying masked language model for sentiment transfer. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI19, pages 5271–5277. International Joint Conferences on Artificial Intelligence Organization.
- Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, and Wenjie Li. 2018. Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 979–988, Melbourne, Australia. Association for Computational Linguistics.
- Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg-Kirkpatrick. 2018. Unsupervised text style transfer using language models as discriminators. In Advances in Neural Information Processing Systems, pages 7287–7298.
- Junbo Zhao, Yoon Kim, Kelly Zhang, Alexander Rush, and Yann LeCun. 2018. Adversarially regularized autoencoders. volume 80 of Proceedings of Machine Learning Research, pages 5902–5911, Stockholmsmassan, Stockholm Sweden. PMLR.
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223–2232.
- Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Advances in neural information processing systems, pages 6830–6841.