Major progress is being recorded regularly on both the technology and exploitation of automatic speech recognition (ASR) and spoken language systems. However, there are still technological barriers to flexible solutions and user satisfaction under some circumstances. This is related to several factors, such as the sensitivity to the environment (background noise), or the weak representation of grammatical and semantic knowledge. Current research is also emphasizing deficiencies in dealing with variation naturally present in speech. For instance, the lack of robustness to foreign accents precludes the use by specific populations. Also, some applications, like directory assistance, particularly stress the core recognition technology due to the very high active vocabulary (application perplexity). There are actually many factors affecting the speech realization: regional, sociolinguistic, or related to the environment or the speaker herself. These create a wide range of variations that may not be modeled correctly (speaker, gender, speaking rate, vocal effort, regional accent, speaking style, non-stationarity, etc.), especially when resources for system training are scarce. This paper outlines current advances related to these topics.

correspondence: Corresponding author. Tel.: +04 93 00 26 28; fax: +04 93 00 26 27. (Wellekens, Christian)

This paper briefly reviews state of the art related to the topic of speech variability sources in automatic speech recognition systems. It focuses on some variations within the speech signal that make the ASR task difficult. The variations detailed in the paper are intrinsic to the speech and affect the different levels of the ASR processing chain. For different sources of speech variation, the paper summarizes the current knowledge and highlights specific feature extraction or modeling weaknesses and current trends

Most voice driven applications are based on recognition grammars. In complex applications it is difficult to exactly predict how the users will formulate their requests even if a careful study of the user's behavior has been performed. Moreover, it is possible that a speaker's word pronunciation does not match the phonetic transcription of the system, mainly in the case of foreign words.Loquendo has developed a tool that collects field data, detects the most significant weaknesses of the application due to pronunciation of formulation mismatches, and filters the collected field corpora. This permits the application designers to perform their analysis only on a reasonable amount of preprocessed and automatically labeled data.This paper presents the approaches that have been devised to detect pronunciation variants of vocabulary words and linguistic formulations not covered by the recognition grammar. Results showing the improvements that have been obtained including automatically detected formulations in three grammars for two languages are also detailed.

Telecom Italia has deployed since the beginning of year 2001 a nationwide automatic Directory Assistance (DA) system that routinely serves customers asking for residential and business listings.

One of the main problems in automatic directory assistance (DA) for business listings is that customers formulate their requests for the same listing with a great variability. We show that an automatic approach allows the detection, from field data, of user formulations that were not foreseen by the designers, and that they can be added, as variants, to the denominations already included in the system to reduce its failures.

Isolated word speech recognizers with fixed vocabularies are often used to provide vocal services through the telephone line. The paper illustrates a simple postprocessing approach that allows the hypotheses produced by a Hidden Markov Model recognizer to be rescored taking into account the global temporal structure of the pronounced words. Our approach does not directly rely on state/word duration modeling. It models, instead, the global time variations of the spectral features of each word and their correlation in time: two important perceptual cues that are only partially exploited by standard HMMs.This method has been evaluated using three isolated word speaker independent systems with vocabulary of different size and complexity. We show that, with minimal overhead, the recognition performance improves not only for small vocabulary recognition systems such as the isolated digit one, or for the recognition of 26 Italian spelling names, but also for a system with a 475 city name vocabulary included in a vocal service that provides information about the main railway connections.

The paper illustrates a search strategy for continuous speech recognition based on the recently developed fast segmental Viterbi algorithm (FSVA), a new search strategy particularly effective for very large vocabulary word recognition. The FSVA search has been extended to deal with continuous speech using a network that merges a general lexical tree and a set of bigram subtrees generated on demand during the search. Results are given for a 751-words speaker independent spontaneous speech recognizer of a railway timetable inquiry application, managed by a dialog system. Preliminary tests have been performed on the Wall Street Journal 5K words 1992 evaluation set.

The paper presents a fast segmental Viterbi algorithm. A new search strategy particularly effective for very large vocabulary word recognition. It performs a tree based, time synchronous, left-to-right beam search that develops time-dependent acoustic and phonetic hypotheses. At any given time, it makes active a sub-word unit associated to an arc of a lexical tree only if that time is likely to be the boundary between the current and the next unit. This new technique, tested with a vocabulary of 188892 directory entries, achieves the same results obtained with the Viterbi algorithm, with a 35% speedup. Results are also presented for a 718 word, speaker independent continuous speech recognition task.

The paper describes a search procedure that, given a set of alternate models for each word of a small vocabulary isolated words recognizer, selects the set of models that minimizes the expected number of errors. The reported results show that the number of errors that occur on the test set by using the best set of models selected from the training set is less than the one achieved by models with a fixed number of states, or the same number of errors is obtained with less states

The partial distance search algorithm (PDS) introduced for reducing the computational complexity of the nearest neighbor search in vector quantization is analyzed. In particular, a detailed analysis of the computational savings that can be obtained by minor modifications to this algorithm is performed. A dynamic programming procedure is proposed that automatically determines how often the comparison with the current minimum distance has to be done in order to minimize the expected global cost of the search. The number and position of the comparisons within the distance evaluation loop depend on the ratio of the cost of a comparison operation to that of the partial distance evaluation. It is shown that the two costs are comparable for RISC (reduced instruction set computer) processors, and a 25% speedup with respect to the PDS algorithm is reported for 24 dimension feature vectors used in a continuous-density HMM (hidden Markov model) system with 16 Gaussian mixtures per state.<>

A parser for continuous speech has to deal with lattices where the word hypotheses of the correct sentence are not usually perfectly aligned and short function words may be missing. To cope with these problems, a two-way interaction between the recognition module and the parser, called feedback verification procedure (FVP), has been investigated. The parser generates many solutions, that are fed back to the recognizer which realigns them against the acoustical data, finds the missing function words among the given candidates, and attributes them a new score. The best scoring solution is finally selected by the parser. Results on a 787-word, speaker-independent, telephone-bandwidth continuous speech recognition task are presented.