AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
A: Hi! Most of my friends work for Google B: do you have anyone who went to columbia? A: Hello? A: I have Jessica a friend of mine A: and Josh, both went to columbia B: or anyone working at apple? B: SELECT A: SELECT of systems, we focus on a symmetric collaborative dialogue sett...

Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings.

meeting of the association for computational linguistics, (2017)

Cited by: 122|Views337
EI

Abstract

We study a symmetric collaborative dialogue setting in which two agents, each with private knowledge, must strategically communicate to achieve a common goal. The open-ended dialogue state in this setting poses new challenges for existing dialogue systems. We collected a dataset of 11K human-human dialogues, which exhibits interesting lex...More

Code:

Data:

0
Introduction
Highlights
  • Current task-oriented dialogue systems (Young et al, 2013; Wen et al, 2017; Dhingra et al, 2017) require a pre-defined dialogue state and a fixed set of dialogue acts
  • A: Hi! Most of my friends work for Google B: do you have anyone who went to columbia? A: Hello? A: I have Jessica a friend of mine A: and Josh, both went to columbia B: or anyone working at apple? B: SELECT (Jessica, Columbia, Computer Science, Google) A: SELECT (Jessica, Columbia, Computer Science, Google) of systems, we focus on a symmetric collaborative dialogue setting, which is task-oriented but encourages open-ended dialogue acts
  • It is hard to define a structured state that captures the diverse semantics in many utterances. To model both structured and open-ended context, we propose the Dynamic Knowledge Graph Network (DynoNet), in which the dialogue state is modeled as a knowledge graph with an embedding Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1766–1776 Vancouver, Canada, July 30 - August 4, 2017. c 2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1162 for each node (Section 3)
  • There has been a recent surge of interest in end-to-end task-oriented dialogue systems, though progress has been limited by the size of available datasets (Serban et al, 2015a)
  • Most work focuses on information-querying tasks, using Wizard-ofOz data collection (Williams et al, 2016; Asri et al, 2016) or simulators (Bordes and Weston, 2017; Li et al, 2016d), In contrast, collaborative dialogues are easy to collect as natural human conversations, and are challenging enough given the large number of scenarios and diverse conversation phenomena
  • Williams et al (2017) use an Long Short Term Memory to automatically infer the dialogue state, but as they focus on dialogue control rather than the full problem, the response is modeled as a templated action, which restricts the generation of richer utterances
Methods
  • The authors compare the model with a rule-based system and a baseline neural model.
  • Both automatic and human evaluations are conducted to test the models in terms of fluency, correctness, cooperation, and human-likeness.
  • The authors randomly split the data into train, dev, and test sets (8:1:1).
  • The authors sequentially sample from the output distribution with a softmax temperature of 0.5.11 Hyperparameters are tuned on the dev set
Results
  • The authors test the systems in two interactive settings: bot-bot chat and bot-human chat. The authors perform both automatic evaluation and human evaluation.

    Automatic Evaluation.
  • The authors test the systems in two interactive settings: bot-bot chat and bot-human chat.
  • The authors perform both automatic evaluation and human evaluation.
  • Automatic Evaluation.
  • The authors compute the cross-entropy (`) of a model on test data.
  • The authors have a model chat with itself on the scenarios from the test set.12.
  • The authors evaluate the chats with respect to language variation, effectiveness, and strategy
  • The authors have a model chat with itself on the scenarios from the test set.12 The authors evaluate the chats with respect to language variation, effectiveness, and strategy
Conclusion
Tables
  • Table1: Main utterance types and examples. We show both standard utterances whose meaning can be represented by simple logical forms (e.g., ask(indoor)), and open-ended ones which require more complex logical forms (difficult parts in bold). Text spans corresponding to entities are underlined
  • Table2: Communication phenomena in the dataset. Evident parts is in bold and text spans corresponding to an entity are underlined. For coreference, the antecedent is in parentheses
  • Table3: Statistics of the MutualFriends dataset
  • Table4: Automatic evaluation on human-human and bot-bot chats on test scenarios. We use " / # to indicate that higher / lower values are better; otherwise the objective is to match humans’ statistics. Best results (except Human) are in bold. Neural models generate shorter (lower Lu) but more diverse (higher H) utterances. Overall, their distributions of utterance types match those of the humans’. (We only show the most frequent speech acts therefore the numbers do not sum to 1.) Rule is effective in completing the task (higher CS), but it is not information-efficient given the large number of attributes (#Attr) and entities (#Ent) mentioned
  • Table5: Results on human-bot/human chats. Best results (except Human) in each column are in bold. We report the average ratings of each system. For third-party evaluation, we first take mean of each question then average the ratings. DynoNet has the best partner satisfaction in terms of fluency (Flnt), correctness (Crct), cooperation (Coop), human likeness (Human). The superscript of a result indicates that its advantage over other systems (r: Rule, s: StanoNet, d: DynoNet) is statistically significant with p < 0.05 given by paired t-tests
  • Table6: Examples of human-bot chats. The mutual friend is highlighted in blue in each KB. Bots’ utterances are in bold and selected items are represented by item IDs. Only the first half of the humanRule chat is shown due to limited space. Multiple utterances of one agent rae separated by ||
  • Table7: Ablations of our model on the dev set show the importance of entity abstraction and message passing (K = 2)
Download tables as Excel
Funding
  • This work is supported by DARPA Communicating with Computers (CwC) program under ARO prime contract no
Reference
  • S. Afantenos, N. Asher, F. Benamara, A. Cadilhac, C. Degremont, P. Denis, M. Guhe, S. Keizer, A. Lascarides, O. Lemon, P. Muller, S. Paul, V. Rieser, and L. Vieu. 2012. Developing a corpus of strategic conversation in the settlers of catan. In SeineDial 2012 The 16th Workshop on the Semantics and Pragmatics of Dialogue.
    Google ScholarFindings
  • L. E. Asri, H. Schulz, S. Sharma, J. Zumer, J. Harris, E. Fine, R. Mehrotra, and K. Suleman. 2016. Frames: A corpus for adding memory to goaloriented dialogue systems. Maluuba Technical Report.
    Google ScholarFindings
  • D. Bahdanau, K. Cho, and Y. Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR).
    Google ScholarLocate open access versionFindings
  • A. Bordes and J. Weston. 2017. Learning end-to-end goal-oriented dialog. In International Conference on Learning Representations (ICLR).
    Google ScholarLocate open access versionFindings
  • B. Dhingra, L. Li, X. Li, J. Gao, Y. Chen, F. Ahmed, and L. Deng. 2017. End-to-end reinforcement learning of dialogue agents for information access. In Association for Computational Linguistics (ACL).
    Google ScholarLocate open access versionFindings
  • J. Duchi, E. Hazan, and Y. Singer. 2010. Adaptive subgradient methods for online learning and stochastic optimization. In Conference on Learning Theory (COLT).
    Google ScholarLocate open access versionFindings
  • M. Henaff, J. Weston, A. Szlam, A. Bordes, and Y. LeCun. 201Tracking the world state with recurrent entity networks. In International Conference on Learning Representations (ICLR).
    Google ScholarLocate open access versionFindings
  • E. Ivanovic. 2005. Dialogue act tagging for instant messaging chat sessions. In Association for Computational Linguistics (ACL).
    Google ScholarLocate open access versionFindings
  • R. Jia and P. Liang. 2016. Data recombination for neural semantic parsing. In Association for Computational Linguistics (ACL).
    Google ScholarLocate open access versionFindings
  • S. Keizer, M. Guhe, H. Cuayahuitl, I. Efstathiou, K. Engelbrecht, M. Dobre, A. Lascarides, and O. Lemon. 2017. Evaluating persuasion strategies and deep reinforcement learning methods for negotiation dialogue agents. In European Association for Computational Linguistics (EACL).
    Google ScholarLocate open access versionFindings
  • C. Kiddon, L. S. Zettlemoyer, and Y. Choi. 2016. Globally coherent text generation with neural checklist models. In Empirical Methods in Natural Language Processing (EMNLP).
    Google ScholarLocate open access versionFindings
  • J. Li, M. Galley, C. Brockett, J. Gao, and B. Dolan. 2016a. A persona-based neural conversation model. In Association for Computational Linguistics (ACL).
    Google ScholarLocate open access versionFindings
  • J. Li, M. Galley, C. Brockett, J. Gao, and W. B. Dolan. 2016b. A diversity-promoting objective function for neural conversation models. In Human Language Technology and North American Association for Computational Linguistics (HLT/NAACL).
    Google ScholarLocate open access versionFindings
  • J. Li, W. Monroe, A. Ritter, D. Jurafsky, M. Galley, and J. Gao. 2016c. Deep reinforcement learning for dialogue generation. In Empirical Methods in Natural Language Processing (EMNLP).
    Google ScholarLocate open access versionFindings
  • X. Li, Z. C. Lipton, B. Dhingra, L. Li, J. Gao, and Y. Chen. 2016d. A user simulator for taskcompletion dialogues. arXiv.
    Google ScholarFindings
  • C. Liu, R. Lowe, I. V. Serban, M. Noseworthy, L. Charlin, and J. Pineau. 20How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Empirical Methods in Natural Language Processing (EMNLP).
    Google ScholarLocate open access versionFindings
  • R. T. Lowe, N. Pow, I. Serban, L. Charlin, C. Liu, and J. Pineau. 20Training End-to-End dialogue systems with the ubuntu dialogue corpus. Dialogue and Discourse 8.
    Google ScholarLocate open access versionFindings
  • H. Mei, M. Bansal, and M. R. Walter. 2016. What to talk about and how? selective generation using LSTMs with coarse-to-fine alignment. In Human Language Technology and North American Association for Computational Linguistics (HLT/NAACL).
    Google ScholarLocate open access versionFindings
  • H. Mei, M. Bansal, and M. R. Walter. 2017. Coherent dialogue with attention-based language models. In Association for the Advancement of Artificial Intelligence (AAAI).
    Google ScholarLocate open access versionFindings
  • C. Potts. 2012. Goal-driven answers in the Cards dialogue corpus. In Proceedings of the 30th West Coast Conference on Formal Linguistics.
    Google ScholarLocate open access versionFindings
  • I. Serban, T. Klinger, G. Tesauro, K. Talamadupula, B. Zhou, Y. Bengio, and A. C. Courville. 2017a. Multiresolution recurrent neural networks: An application to dialogue response generation. In Association for the Advancement of Artificial Intelligence (AAAI).
    Google ScholarLocate open access versionFindings
  • I. Serban, A. Sordoni, R. Lowe, L. Charlin, J. Pineau, A. C. Courville, and Y. Bengio. 2017b. A hierarchical latent variable encoder-decoder model for generating dialogues. In Association for the Advancement of Artificial Intelligence (AAAI).
    Google ScholarLocate open access versionFindings
  • I. V. Serban, R. Lowe, L. Charlin, and J. Pineau. 2015a. A survey of available corpora for building data-driven dialogue systems. arXiv preprint arXiv:1512.05742.
    Findings
  • I. V. Serban, A. Sordoni, Y. Bengio, A. Courville, and J. Pineau. 2015b. Building end-to-end dialogue systems using generative hierarchical neural network models. arXiv preprint arXiv:1507.04808.
    Findings
  • L. Shang, Z. Lu, and H. Li. 2015. Neural responding machine for short-text conversation. In Association for Computational Linguistics (ACL).
    Google ScholarLocate open access versionFindings
  • A. Sordoni, M. Galley, M. Auli, C. Brockett, Y. Ji, M. Mitchell, J. Nie, J. Gao, and B. Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. In North American Association for Computational Linguistics (NAACL).
    Google ScholarLocate open access versionFindings
  • P. Su, M. Gasic, N. Mrksic, L. M. Rojas-Barahona, S. Ultes, D. Vandyke, T. Wen, and S. J. Young. 2016. Continuously learning neural dialogue management. arXiv preprint arXiv:1606.02689.
    Findings
  • A. Vogel, M. Bodoia, C. Potts, and D. Jurafsky. 2013. Emergence of gricean maxims from multi-agent decision theory. In North American Association for Computational Linguistics (NAACL). pages 1072– 1081.
    Google ScholarLocate open access versionFindings
  • T. Wen, M. Gasic, N. Mrksic, L. M. Rojas-Barahona, P. Su, S. Ultes, D. Vandyke, and S. Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In European Association for Computational Linguistics (EACL).
    Google ScholarLocate open access versionFindings
  • J. D. Williams, K. Asadi, and G. Zweig. 2017. Hybrid code networks: Practical and efficient end-toend dialog control with supervised and reinforcement learning. In Association for Computational Linguistics (ACL).
    Google ScholarLocate open access versionFindings
  • J. D. Williams, A. Raux, and M. Henderson. 2016. The dialog state tracking challenge series: A review. Dialogue and Discourse 7.
    Google ScholarLocate open access versionFindings
  • J. D. Williams and S. Young. 2007. Partially observable Markov decision processes for spoken dialog systems. Computer Speech & Language 21(2):393– 422.
    Google ScholarLocate open access versionFindings
  • S. Young, M. Gasic, B. Thomson, and J. D. Williams. 2013. POMDP-based statistical spoken dialog systems: A review. Proceedings of the IEEE 101(5):1160–1179.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科