Score-Based Generative Modeling through Stochastic Differential Equations

international conference on learning representations, 2020.

Cited by: 0|Bibtex|Views18
Other Links: arxiv.org|academic.microsoft.com
Weibo:
A general framework for training and sampling from score-based models that unifies and generalizes previous methods, allows likelihood computation, and enables controllable generation.

Abstract:

Creating noise from data is easy; creating data from noise is generative modeling. We present a stochastic differential equation (SDE) that smoothly transforms a complex data distribution to a known prior distribution by slowly injecting noise, and a corresponding reverse-time SDE that transforms the prior distribution back into the dat...More

Code:

Data:

0
Introduction
  • The goal of task-oriented dialog systems is to assist the user in completing a certain task by performing an action or retrieving relevant information (Tur & Mori, 2011).
  • CSP has been extensively studied in several academic and industrial research settings such as dialog systems (e.g., dialog state tracking in MWOZ (Budzianowski et al, 2018)), interacting with physical agents (e.g., (Chai et al, 2018)), context-dependent semantic parsing (e.g., SPARC (Yu et al, 2019b)), SQL-grounded state tracking (e.g., COSQL (Yu et al, 2019a)), and sequential question answering (e.g., SQA (Iyyer et al, 2017))
  • These settings differ in some respect, but they share the same overall objective and key challenge: how to jointly represent the natural language utterances and underlying structured ontology while taking into consideration the multi-turn dynamics of the dialog.
  • Open-domain dialogue language models such as DialoGPT (Zhang et al, 2020) and ConveRT (Henderson et al, 2019) are pre-trained on the Reddit data and applied to dialog response generation and retrieval tasks
Highlights
  • The goal of task-oriented dialog systems is to assist the user in completing a certain task by performing an action or retrieving relevant information (Tur & Mori, 2011)
  • In contrast to open-domain dialogs, Conversational Semantic Parsing (CSP) datasets are usually much smaller due to the difficulty and expense of obtaining and labeling data. unlike most prior work on contextualized LMs which are pre-trained on free text, according to the finding where questions in CSP tasks are more compositional than other free-text since they can be mapped into formal representations, we propose to train SCORE on synthesized conversational semantic parsing data with multiple training objectives that aim to ground utterances into the schema of the underlying ontology and to model the relationship between different utterances in the multi-turn conversation
  • We demonstrate that: (1) SCORE training objectives can effectively incorporate synthesized data, (2) a single pre-trained SCORE model can be used for several CSP tasks and can be combined with many baseline systems with different model architectures and (3) SCORE significantly improve all baseline systems and achieves new state-of-the-art results on three benchmarks (SPARC, SPARC, and MWOZ) and comparable performance to state-of-the-art results on the fourth (SQA)
  • We presented SCORE a new pre-training approach for conversational semantic parsing
  • The training objectives of SCORE aim to induce natural language representations that capture the multi-turn dynamics, compositional semantic of the target language, and the references to the structural ontology appearing in the dialog
  • As Table 4 shows, SCORE improves question match accuracy (QM) by 2.6% and interaction match accuracy (IM) by 4.9% over ROBERTA with Wang et al (2019) as the base model
  • Our empirical results on four different CSP tasks demonstrated that SCORE can be used to significantly improve the performance of existing strong baseline models by replacing an existing pre-trained LM with our SCORE pre-trained model
Results
  • RESULTS AND ANALYSIS

    Overall Results The results of SPARC and COSQL, MWOZ, and SQA are in Table 2, 3, and 4 respectively.
  • We directly concatenate the additional SPARC examples to COSQL training set, and train RAT-SQL+ROBERTA on it, which slightly improves the performance (19.6% vs 19.3%) but not as large as SCORE (22.0% vs 19.3%).’ because SQA is weakly-supervised sequential question answering, which differs from SPARC, we first fine-tune ROBERTA on the additional SPARC examples using CCS, and apply it to SQA.
  • As for GPT-2, Wu et al (2020) and Hosseini-Asl et al (2020) found it does not outperform BERT on MWOZ
Conclusion
  • The authors presented SCORE a new pre-training approach for conversational semantic parsing. The training objectives of SCORE aim to induce natural language representations that capture the multi-turn dynamics, compositional semantic of the target language, and the references to the structural ontology appearing in the dialog.
  • SCORE can be used with many semantic parsing models as a drop-in replacement for general pretrained LMs. The authors demonstrated SCORE effectiveness by using it as a feature representation encoder with strong baseline models for a wide range of CSP tasks.
  • The authors' empirical results on four different CSP tasks demonstrated that SCORE can be used to significantly improve the performance of existing strong baseline models by replacing an existing pre-trained LM with the SCORE pre-trained model.
  • The authors hope SCORE will encourage further exploration of the benefits and limitations of pre-training approaches for CSP systems
Summary
  • Introduction:

    The goal of task-oriented dialog systems is to assist the user in completing a certain task by performing an action or retrieving relevant information (Tur & Mori, 2011).
  • CSP has been extensively studied in several academic and industrial research settings such as dialog systems (e.g., dialog state tracking in MWOZ (Budzianowski et al, 2018)), interacting with physical agents (e.g., (Chai et al, 2018)), context-dependent semantic parsing (e.g., SPARC (Yu et al, 2019b)), SQL-grounded state tracking (e.g., COSQL (Yu et al, 2019a)), and sequential question answering (e.g., SQA (Iyyer et al, 2017))
  • These settings differ in some respect, but they share the same overall objective and key challenge: how to jointly represent the natural language utterances and underlying structured ontology while taking into consideration the multi-turn dynamics of the dialog.
  • Open-domain dialogue language models such as DialoGPT (Zhang et al, 2020) and ConveRT (Henderson et al, 2019) are pre-trained on the Reddit data and applied to dialog response generation and retrieval tasks
  • Results:

    RESULTS AND ANALYSIS

    Overall Results The results of SPARC and COSQL, MWOZ, and SQA are in Table 2, 3, and 4 respectively.
  • We directly concatenate the additional SPARC examples to COSQL training set, and train RAT-SQL+ROBERTA on it, which slightly improves the performance (19.6% vs 19.3%) but not as large as SCORE (22.0% vs 19.3%).’ because SQA is weakly-supervised sequential question answering, which differs from SPARC, we first fine-tune ROBERTA on the additional SPARC examples using CCS, and apply it to SQA.
  • As for GPT-2, Wu et al (2020) and Hosseini-Asl et al (2020) found it does not outperform BERT on MWOZ
  • Conclusion:

    The authors presented SCORE a new pre-training approach for conversational semantic parsing. The training objectives of SCORE aim to induce natural language representations that capture the multi-turn dynamics, compositional semantic of the target language, and the references to the structural ontology appearing in the dialog.
  • SCORE can be used with many semantic parsing models as a drop-in replacement for general pretrained LMs. The authors demonstrated SCORE effectiveness by using it as a feature representation encoder with strong baseline models for a wide range of CSP tasks.
  • The authors' empirical results on four different CSP tasks demonstrated that SCORE can be used to significantly improve the performance of existing strong baseline models by replacing an existing pre-trained LM with the SCORE pre-trained model.
  • The authors hope SCORE will encourage further exploration of the benefits and limitations of pre-training approaches for CSP systems
Tables
  • Table1: Comparison of CSP datasets. Examples from two of the datasets are shown in Figure 1. Cross-domain means the train and test sets have different domains, so MWOZ is not cross-domain. and column names, slots, etc.) of the target database (ontology) d. To cover different variants of the problem, we consider four popular CSP tasks shown in Table 1: SPARC (sequential text-toSQL), COSQL (conversational text-to-SQL), MWOZ (dialogue state tracking), and SQA (weakly supervised sequential question answering). They have different target formal language and structured ontology: • For the utterance u, it is the user question for SPARC and SQA, while for COSQL and MWOZ, u is the combination of a user query and a system response. • For the database d, SPARC and COSQL use multi-table databases; for MWOZ, the pre-defined ontology d can also be viewed as a database; for SQA, d is a single table. • For the formal representation q, it is the SQL query for SPARC and COSQL; in MWOZ it is the slot-value pairs that can be viewed as simple SQL queries consisting of SELECT and WHERE clauses; and for SQA, q is the latent program. Data statistics for task-oriented dialogue pre-training
  • Table2: The SPARC and COSQL accuracy over all questions (QM) and all interactions (IM). The scores of IGSQL + BERT and R2SQL + BERT are from the official leaderboards
  • Table3: Joint goal accuracies (JGA) on MWOZ 2.1 test set. All models use a BERT-like encoder/GPT
  • Table4: Question (QM) and interaction (IM) accuracy on the SQA test set
  • Table5: The effect of SCORE pre-training objectives. Improvements are shown in the parentheses
  • Table6: Detailed results on the dev set of SPARC. Qi is the accuracy of the ith conversation question
  • Table7: Effect of synthetic data as training data augmentation
  • Table8: Performance of SCORE pre-trained on different synthesized data on MWOZ
  • Table9: Performance of on SQA when only 10% of training data is available. We choose SQA because its annotation is most different from the synthetic text-to-SQL
  • Table10: Detailed results of COSQL on the dev set. Qi is the accuracy of the ith question in the conversation
  • Table11: Detailed results of SQA on the test set. Qi is the accuracy of the ith question in the conversation
  • Table12: An example of synthetic conversational text-to-SQL data
Download tables as Excel
Related work
  • Conversational Semantic Parsing Conversational semantic parsing is one of the most important research topics in conversational AI and has been studied in different settings including task-oriented dialogues, question answering, and text-to-SQL. Task-oriented dialog systems (Henderson et al., 2014; Wen et al, 2016; Mrksicet al., 2017; Budzianowski et al, 2018) aim to help users accomplish a specific task (e.g. flight booking) and often pre-define slot templates grounded in a domainspecific ontology. In comparison, several other datasets were recently introduced for cross-domain conversational text-to-SQL tasks (SPARC and COSQL (Yu et al, 2019a;b)) and sequential questions answers over tables (Iyyer et al, 2017). While the previous work has achieved significant progress in different datasets separately, to the best of our knowledge, we are the first to study four different CSP tasks together (sequential text-to-SQL, conversational text-to-SQL, dialog state tracking, and weakly-supervised sequential question answering) by addressing the shared key challenge of learning representations in pre-trained language models that capture the alignment between the dialogue flow and the structural context. Conversational Language Model Pre-training Several recent efforts have demonstrated the value of adapting pre-trained LMs to specific tasks using different pre-training objectives, e.g., summarization (Zhang et al, 2019b), knowledge inference (Sun et al, 2019b; Liu et al, 2019a), etc. Closest to our work is adapting pre-trained LMs for open-domain chit-chat models and for tabular data representation. The former focuses on improving response generation on open-ended dialogues by adding a pre-training step on open-domain conversations data, such as Reddit data (Zhang et al, 2020; Henderson et al, 2019). For example, Wu et al (2020) introduced ToD-BERT, a pre-trained language model combining 9 high-quality human-human task-oriented dialogue datasets to conduct language model and response selection pre-training. However, they use language modeling training objectives over free-form text and therefore have limited ability to represent structural data. The latter has focused on improving language model pre-training for encoding tabular data (Yin et al, 2020; Herzig et al, 2020), but they focus on the single turn semantic parsing setting. Our approach is different from previous work because we address the challenge of conversational semantic parsing tasks by learning pretrained representation for both the multi-turn dynamics of the dialog and the relation between the unstructured language utterance and the structured ontology. Furthermore, our pre-training approach is much more data-efficient than prior LM pre-training work and saves a lot of time and computing resources (Appendix D for more details). Our pre-training step can be done within only one day using 8 V100 GPUs. Using Synthesized Data for Semantic Parsing Synthesized data has been frequently used in semantic parsing to alleviate the challenge of labeled data scarcity. For example, Wang et al (2015) proposed a method for training semantic parsers in new domains by generating logical forms and canonical utterances and then paraphrasing the canonical utterances via crowd-sourcing. Similar approaches were used to train semantic parsers in other domains and settings (Zhong et al, 2017; Su et al, 2017; Cheng et al, 2018; Shah et al, 2018). Another line of work has proposed using synthesized data to adapt single turn semantic parsing models to new domains (Jia & Liang, 2016; Yoo et al, 2018; Campagna et al, 2019) and task-oriented dialogues (Campagna et al, 2020). However, they reported that combining synthetic data and the supervised data does not yield significant improvements, consistent with results by Herzig et al (2020). By contrast, we introduce a new data synthesize procedure for conversational text-to-SQL dialogues and use it in a different way by pretraining language models to induce better representations for many CSP tasks. Our synthesized data can be easily generated without human involvement and the pre-trained models add value to different tasks simultaneously.
Funding
  • As Table 4 shows, SCORE improves QM by 2.6% and IM by 4.9% over ROBERTA with Wang et al (2019) as the base model
  • Pre-training on the synthesized data with our objectives improves the performance on the downstream tasks
  • Table 9 demonstrates that SCORE delivers even larger improvements compared to the ROBERTA baseline when only 10% training data is available (3.8% vs 2.6%)
  • Furthermore, we are able to achieve state-of-the-art results on three of these tasks
Study subjects and analysis
people: 8
Usr: I am looking for a cheap restaurant in the centre of the city Sys: There is a cheap chinese restaurant called Dojo Noodle Bar. Usr: Yes please , for 8 people at 18:30 on Thursday. SELECT title FROM book ORDER BY sale_amount DESC LIMIT 3

people: 8
The TCS objective aims to capture this Dojo Noodle Bar. Usr: Yes please , for 8 people at 18:30 on Thursday grounding of conte...x.t.. flow. To this end, it targets predicting the difference in formal Taxi: leaveAt |...|rdeestpinartieonsentations between dialog turns based on the natural language utterance

task-oriented dialog datasets: 9
Importantly for regularization, we only apply this loss on in-domain human-annotated natural language data. Namely, it includes utterances in SPARC, COSQL, and SQA as well as nine task-oriented dialog datasets processed by Wu et al (2020) for MWOZ (see data statistics in Figure 4). Formally, the MLM loss is given by: LMLM(Ct) = CrossEntropyVocab(LayerNorm(W4hm t ))

Reference
  • Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. ArXiv, abs/1607.06450, 2016.
    Findings
  • Chandra Bhagavatula, Thanapon Noraset, and Doug Downey. Tabel: Entity linking in web tables. In International Semantic Web Conference, 2015.
    Google ScholarLocate open access versionFindings
  • Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.
    Findings
  • Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. Multiwoz - a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In EMNLP, 2018.
    Google ScholarLocate open access versionFindings
  • Giovanni Campagna, Silei Xu, Mehrad Moradshahi, Richard Socher, and Monica S. Lam. Genie: A generator of natural language semantic parsers for virtual assistant commands. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 394–410. Association for Computing Machinery, 2019.
    Google ScholarLocate open access versionFindings
  • Giovanni Campagna, Agata Foryciarz, Mehrad Moradshahi, and Monica S. Lam. Zero-shot transfer learning with synthesized data for multi-domain dialogue state tracking. In Proceedings of 58th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2020.
    Google ScholarLocate open access versionFindings
  • Joyce Y Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu. Language to action: Towards interactive task learning with physical agents. In IJCAI, pp. 2–9, 2018.
    Google ScholarLocate open access versionFindings
  • Jianpeng Cheng, Siva Reddy, and Mirella Lapata. Building a neural semantic parser from a domain ontology. ArXiv, abs/1812.10037, 2018.
    Findings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, 2019.
    Google ScholarLocate open access versionFindings
  • Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyag Gao, and Dilek Hakkani-Tur. Multiwoz 2.1: Multi-domain dialogue state corrections and state tracking baselines. arXiv preprint arXiv:1907.01669, 2019.
    Findings
  • Shuyang Gao, Abhishek Sethi, Sanchit Agarwal, Tagyoung Chung, and Dilek Hakkani-Tur. Dialog state tracking: A neural reading comprehension approach. In SIGDial, 2019.
    Google ScholarLocate open access versionFindings
  • Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, July 2020. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • M. Heck, Carel van Niekerk, Nurul Lubis, Christian Geishauser, Hsien-Chin Lin, M. Moresi, and Milica Gavsi’c. Trippy: A triple copy strategy for value independent neural dialog state tracking. In SIGdial, 2020.
    Google ScholarLocate open access versionFindings
  • Matthew Henderson, Blaise Thomson, and Jason D. Williams. The second dialog state tracking challenge. In SIGDIAL Conference, 2014.
    Google ScholarLocate open access versionFindings
  • Matthew Henderson, Inigo Casanueva, Nikola Mrkvsi’c, P. Su, Tsung-Hsien, and Ivan Vulic. Convert: Efficient and accurate conversational representations from transformers. ArXiv, abs/1911.03688, 2019.
    Findings
  • Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. ArXiv, abs/1610.02136, 2016.
    Findings
  • Jonathan Herzig, P. Nowak, Thomas Muller, Francesco Piccinno, and Julian Martin Eisenschlos. Tapas: Weakly supervised table parsing via pre-training. In ACL, 2020.
    Google ScholarFindings
  • Ehsan Hosseini-Asl, B. McCann, Chien-Sheng Wu, Semih Yavuz, and R. Socher. A simple language model for task-oriented dialogue. ArXiv, abs/2005.00796, 2020.
    Findings
  • Wonseok Hwang, Jinyeung Yim, Seunghyun Park, and Minjoon Seo. A comprehensive exploration on wikisql with table-aware word contextualization. ArXiv, abs/1902.01069, 2019.
    Findings
  • Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang. Search-based neural structured learning for sequential question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada, July 2017. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Sina Jafarpour, Christopher JC Burges, and Alan Ritter. Filter, rank, and transfer the knowledge: Learning to chat. Advances in Ranking, 10:2329–9290, 2010.
    Google ScholarLocate open access versionFindings
  • Robin Jia and Percy Liang. Data recombination for neural semantic parsing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016.
    Google ScholarLocate open access versionFindings
  • Sung-Dong Kim, Sohee Yang, Gyuwan Kim, and S. Lee. Efficient dialogue state tracking by selectively overwriting memory. In ACL, 2020.
    Google ScholarLocate open access versionFindings
  • Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014.
    Findings
  • Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, July 2020. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Q. Liu, B. Chen, Jiaqi Guo, Jian-Guang Lou, Bin Zhou, and Dongmei Zhang. How far are we from effective context modeling ? an exploratory study on semantic parsing in context. ArXiv, abs/2002.00652, 2020.
    Findings
  • Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. K-bert: Enabling language representation with knowledge graph, 2019a.
    Google ScholarFindings
  • Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke S. Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692, 2019b.
    Findings
  • Nikola Mrksic, Diarmuid O Seaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1777–1788. Association for Computational Linguistics, 2017.
    Google ScholarLocate open access versionFindings
  • Thomas Muller, Francesco Piccinno, Massimo Nicosia, Peter Shaw, and Yasemin Altun. Answering conversational questions on structured data without logical forms. In EMNLP/IJCNLP, 2019.
    Google ScholarLocate open access versionFindings
  • Arvind Neelakantan, Quoc V. Le, M. Abadi, A. McCallum, and Dario Amodei. Learning a natural language interface with neural programmer. ArXiv, abs/1611.08945, 2017.
    Findings
  • Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, pp. 1470–1480, 2015.
    Google ScholarLocate open access versionFindings
  • Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 2019.
    Google ScholarLocate open access versionFindings
  • Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020.
    Google ScholarLocate open access versionFindings
  • Alan Ritter, Colin Cherry, and William B. Dolan. Data-driven response generation in social media. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 583–593, Edinburgh, Scotland, UK., July 2011. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Ohad Rubin and Jonathan Berant. Smbop: Semi-autoregressive bottom-up semantic parsing. arXiv preprint arXiv:2010.12412, 2020.
    Findings
  • Timo Schick and Hinrich Schutze. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.07118, 2020.
    Findings
  • Pararth Shah, Dilek Hakkani-Tur, Gokhan Tur, Abhinav Rastogi, Ankur Bapna, Neha Nayak, and Larry Heck. Building a conversational agent overnight with dialogue self-play, 2018.
    Google ScholarFindings
  • Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Peter Shaw, Ming-Wei Chang, Panupong Pasupat, and Kristina Toutanova. Compositional generalization and natural language variation: Can a semantic parsing approach handle both? arXiv preprint arXiv:2010.12725, 2020.
    Findings
  • Yu Su, Ahmed Hassan Awadallah, Madian Khabsa, P. Pantel, M. Gamon, and Mark J. Encarnacion. Building natural language interfaces to web apis. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017.
    Google ScholarLocate open access versionFindings
  • Yibo Sun, Duyu Tang, Nan Duan, Jingjing Xu, X. Feng, and B. Qin. Knowledge-aware conversational semantic parsing over web tables. In NLPCC, 2019a.
    Google ScholarLocate open access versionFindings
  • Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. Ernie: Enhanced representation through knowledge integration, 2019b.
    Google ScholarFindings
  • Gokhan Tur and Renato De Mori. Spoken language understanding: Systems for extracting semantic information from speech. John Wiley & Sons, 2011.
    Google ScholarFindings
  • Bailin Wang, Ivan Titov, and Mirella Lapata. Learning semantic parsers from denotations with latent structured alignments and abstract programs. In Proceedings of EMNLP, 2019.
    Google ScholarLocate open access versionFindings
  • Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. RAT-SQL: Relation-aware schema encoding and linking for text-to-SQL parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7567–7578, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.677.
    Locate open access versionFindings
  • Yushi Wang, Jonathan Berant, and Percy Liang. Building a semantic parser overnight. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1332–1342, Beijing, China, July 2015. Association for Computational Linguistics. doi: 10.3115/v1/P15-1129.
    Locate open access versionFindings
  • Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gasic, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562, 2016.
    Findings
  • Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771, 2019.
    Findings
  • Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. Transferable multi-domain state generator for task-oriented dialogue systems. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July 2019. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Chien-Sheng Wu, Steven C.H. Hoi, Richard Socher, and Caiming Xiong. TOD-BERT: Pre-trained natural language understanding for task-oriented dialogue. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
    Google ScholarLocate open access versionFindings
  • Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. TaBERT: Pretraining for joint understanding of textual and tabular data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8413–8426, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.745.
    Locate open access versionFindings
  • Kang Min Yoo, Youhyun Shin, and Sang goo Lee. Data augmentation for spoken language understanding via joint variational generation, 2018.
    Google ScholarFindings
  • Tao Yu, Michihiro Yasunaga, Kai Yang, Rui Zhang, Dongxu Wang, Zifan Li, and Dragomir Radev. Syntaxsqlnet: Syntax tree networks for complex and cross-domain text-to-sql task. In Proceedings of EMNLP. Association for Computational Linguistics, 2018a.
    Google ScholarLocate open access versionFindings
  • Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In EMNLP, 2018b.
    Google ScholarLocate open access versionFindings
  • Tao Yu, Rui Zhang, He Yang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, and Dragomir Radev. Cosql: A conversational text-to-sql challenge towards cross-domain natural language interfaces to databases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, 2019a.
    Google ScholarLocate open access versionFindings
  • Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Irene Li Heyang Er, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Vincent Zhang Jonathan Kraft, Caiming Xiong, Richard Socher, and Dragomir Radev. Sparc: Cross-domain semantic parsing in context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019b. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Jian-Guo Zhang, Kazuma Hashimoto, Chien-Sheng Wu, Yao Wan, Philip S Yu, Richard Socher, and Caiming Xiong. Find or classify? dual strategy for slot-value predictions on multi-domain dialog state tracking. arXiv preprint arXiv:1910.03544, 2019a.
    Findings
  • Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J Liu. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. arXiv preprint arXiv:1912.08777, 2019b.
    Findings
  • Rui Zhang, Tao Yu, He Yang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, and Dragomir Radev. Editing-based sql query generation for cross-domain context-dependent questions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, 2019c.
    Google ScholarLocate open access versionFindings
  • Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL, system demonstration, 2020.
    Google ScholarFindings
  • Victor Zhong, Caiming Xiong, and Richard Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, abs/1709.00103, 2017.
    Findings
  • Victor Zhong, M. Lewis, Sida I. Wang, and Luke Zettlemoyer. Grounded adaptation for zero-shot executable semantic parsing. The 2020 Conference on Empirical Methods in Natural Language Processing, 2020.
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments