The START system responds to natural language queries with answers in text, pictures, and other media. START's sentence-level natural language parsing relies on a number of mechanisms to help it process the huge, diverse resources available on the World Wide Web. Blitz, a hybrid heuristic- and corpus-based natural language preprocessor enables START to integrate a large and ever-changing lexicon of proper names, by using heuristic rules and precompiled tables of symbols to preprocess various highly regular and fixed expressions into lexical tokens. LaMeTH, a content-based system for extracting information from HTML documents, assists START by providing a uniform method of accessing information on the Web in real time. These mechanisms have considerably improved STARTS ability to analyze real-world sentences and answer queries through expansion of its lexicon and integration of Web resources.

The flow of natural language is often broken by constructions which are difficult to analyze with conventional linguistic parsers. To handle these constructions, which include numbers, dates, addresses, etc., and, to a lesser extent, proper nouns, NL systems typically implement specialized new rules. This leads to a level of complexity which renders maintenance or improvement difficult. Analyzing and tokenizing these constructions with an independent preprocessor can alleviate the burden on already taxed systems. Because these constructions have highly regular forms, strict structure, and can be largely understood in the absence of context, it is possible to shift the burden of processing away from the primary parser, and onto a simpler, faster, non-linguistic preprocessor. This paper describes Blitz, a hybrid database- and heuristic-based natural language preprocessor, which has been integrated into the START Natural Language System in order to demonstrate how non-linguistic preprocessing can improve parsing. As a result, START’s ability to analyze real-world sentences has improved considerably. Advantages of Blitz over existing systems are also discussed.

The flow of natural language is often broken by constructions which are difficult to analyze with conventional linguistic parsers. To handle these constructions, which include numbers, dates, addresses, etc., and, to a lesser extent, proper nouns, NL systems typically implement specialized new rules. This leads to a level of complexity which renders maintenance or improvement difficult. Analyzing and tokenizing these constructions with an independent preprocessor can alleviate the burden on already taxed systems. Because these constructions have highly regular forms, strict structure, and can be largely understood in the absence of context, it is possible to shift the burden of processing away from the primary parser, and onto a simpler, faster, non-linguistic preprocessor. This paper describes Blitz, a hybrid database- and heuristic-based natural language preprocessor, which has been integrated into the START Natural Language System in order to demonstrate how non-linguistic preprocessing can improve parsing. As a result, START's ability to analyze real-world sentences has improved considerably. Advantages of Blitz over existing systems are also discussed.