Learning by reading: a prototype system, performance baseline and lessons learned

A traditional goal of Artificial Intelligence research has been a system that can read unrestricted natural language texts on a given topic, build a model of that topic and reason over the model. Natural Language Processing advances in syntax and semantics have made it possible to extract a limited form of meaning from sentences. Knowledge Representation research has shown that it is possible to model and reason over topics in interesting areas of human knowledge. It is useful for these two communities to reunite periodically to see where we stand with respect to the common goal of text understanding. In this paper, we describe a coordinated effort among researchers from the Natural Language and Knowledge Representation and Reasoning communities. We routed the output of existing NL software into existing KR software to extract knowledge from texts for integration with engineered knowledge bases. We tested the system on a suite of roughly 80 small English texts about the form and function of the human heart, as well as a handful of "confuser" texts from other domains. We then manually evaluated the knowledge extracted from novel texts. Our conclusion is that the technology from these fields is mature enough to start producing unified machine reading systems. The results of our exercise provide a performance baseline for systems attempting to acquire models from text.