We present an integrated approach to speech and natural language processing which uses a single parser to create training for a statistical speech recognition component and for interpreting recognized text. On the speech recognition side, our innovation is the use of a statistical model combining N-gram and context-free grammars. On the natural language side, our innovation is the integration of parsing and semantic interpretation to build references for only targeted phrase types. In both components, a semantic grammar and partial parsing facilitate robust processing of the targeted portions of a domain. This integrated approach introduces as much linguistic structure and prior statistical information as is available while maintaining a robust full-coverage statistical language model for recognition. In addition, our approach facilitates both the direct detection of linguistic constituents within the speech recognition algorithms and the creation of semantic interpretations of the recognized phrases.

The objective of this work is automatic, real-time "gisting" of voice traffic for updating of information in databases, for producing timely reports, and for prompt notification of events of interest. Specifically, the goal is to build a prototype, real-time system capable of processing radio communication between air traffic controllers and pilots; identifying dialogs and extracting their "gist" (e.g., identifying flights, determining whether they are landing or taking off), and producing a continuous output stream with that information. The approach is intended to be general and applicable to other domains.

A word-spotting system using Gaussian hidden Markov models is presented. Several aspects of this problem are investigated. Specifically, results are reported on the use of various signal processing and feature transformation techniques. The authors have observed that performance can be greatly affected by the choice of features used, the covariance structure of the Gaussian models, and transformations based on energy and feature distributions. Due to the open-set nature of the problem, the specific techniques for modeling out-of-vocabulary speech and the choice of scoring metric can have a significant effect on performance

Statistical language models have been successfully used to improve the performance of continuous speech recognition algorithms. Application of such techniques is difficult when only a small training corpus is available. The authors present an approach for dealing with limited training available from the DARPA resource management domain. An initial training corpus of sentences was abstracted by replacing sentence fragments or phrases with variables. This training corpus of phrase sequences was used to derive parameters of a Markov model. The probability of a word sequence is then decomposed into the probability of possible phrase sequences within each of the phrases. Initial results obtained on 150 utterances from six speakers in the DARPA database indicate that this language modeling technique has potential for improved recognition performance. Furthermore, this approach provides a framework for incorporating linguistic knowledge into statistical language models