Named entity recognition (NER) is among SLU tasks that usually extract semantic information from textual documents. Until now, NER from speech is made through a pipeline process that consists in processing first an automatic speech recognition (ASR) on the audio and then processing a NER on the ASR outputs. Such approach has some disadvantages (error propagation, metric to tune ASR systems sub-optimal in regards to the final task, reduced space search at the ASR output level, ...) and it is known that more integrated approaches outperform sequential ones, when they can be applied. In this paper, we explore an end-to-end approach that directly extracts named entities from speech, though a unique neural architecture. On a such way, a joint optimization is possible for both ASR and NER. Experiments are carried on French data easily accessible, composed of data distributed in several evaluation campaigns. The results are promising since this end-to-end approach provides similar results (F-measure=0.66 on test data) than a classical pipeline approach to detect named entity categories (F-measure=0.64). Last, we also explore this approach applied to semantic concept extraction, through a slot filling task known as a spoken language understanding problem, and also observe an improvement in comparison to a pipeline approach.

Cet article propose une methode de simulation d'erreurs de systemes de reconnaissance automatique de la parole (SRAP) a partir de transcriptions manuelles, et montre son utilite pour rendre les systemes de comprehension automatique de la parole (SCAP) plus robustes aux erreurs de SRAP. Partant du principe que le SRAP confond les mots acoustiquement et linguistiquement proches, cette methode s'appuie sur l'utilisation de plongements de mots acoustiques et linguistiques pour calculer une mesure de similarite entre les mots : cette mesure vise a predire les confusions de mots faites par le SRAP. Les experiences menees sur le corpus MEDIA (reservations d'hotel) montrent que cette approche ameliore significativement les performances des SCAP avec une reduction relative de 21,2% du taux d'erreur concept/valeur, en particulier quand le SCAP est neuronal (reduction de 22,4%). Une comparaison avec une methode de bruitage naif montre la pertinence de l'approche de bruitage proposee.

This paper presents an approach to simulate automatic speech recognition (ASR) errors from manual transcriptions and describes how it can be used to improve the performance of spoken language understanding (SLU) systems. In particular, we point out that this noising process is very usefull to obtain a more robust SLU system to ASR errors in case of insufficient training data or more if ASR transcriptions are not available during the training of the SLU model. The proposed method is based on the use of both acoustic and linguistic word embeddings in order to define a similarity measure between words dedicated to predict ASR confusions. Actually, we assume that words acoustically and linguistically close are the ones confused by an ASR system. By using this similarity measure in order to randomly substitute correct words by potentially confusing words in manual annotations used to train CRFor neuralbased SLU systems, we augment the training corpus with these new noisy data. Experiments were carried on the French MEDIA corpus focusing on hotel reservation. They show that this approach significantly improves SLU system performance with a relative reduction of 21.2% of concept/value error rate (CVER), particularly when the SLU system is based on a neural approach (reduction of 22.4% of CVER). A comparison to a naive noising approach shows that the proposed noising approach is particularly relevant.

Cet article propose une méthode de simulation d’erreurs de systèmes de reconnaissance automatique de la parole (SRAP) à partir de transcriptions manuelles, et montre son utilité pour rendre les systèmes de compréhension automatique de la parole (SCAP) plus robustes aux erreurs de SRAP. Partant du principe que le SRAP confond les mots acoustiquement et linguistiquement proches, cette méthode s’appuie sur l’utilisation de plongements de mots acoustiques et linguistiques pour calculer une mesure de similarité entre les mots : cette mesure vise à prédire les confusions de mots faites par le SRAP. Les expériences menées sur le corpus MEDIA (réservations d’hôtel) montrent que cette approche améliore significativement les performances des SCAP avec une réduction relative de 21,2% du taux d’erreur concept/valeur, en particulier quand le SCAP est neuronal (réduction de 22,4%). Une comparaison avec une méthode de bruitage naïf montre la pertinence de l’approche de bruitage proposée. This paper presents an approach to simulate automatic speech recognition (ASR) errors from manual transcriptions and how it can be used to improve the performance of spoken language understanding (SLU) systems. The proposed method is based on the use of both acoustic and linguistic word embeddings in order to define a similarity measure between words. This measure is dedicated to predict ASR confusions. Actually, we assume that words acoustically and linguistically close are the ones confused by an ASR system. Experiments were carried on the French MEDIA corpus focusing on hotel reservation. They show that this approach significantly improves SLU system performance with a relative reduction of 21.2% of concept/value error rate (CVER), particularly when the SLU system is based on a neural approach (reduction of 22.4% of CVER). A comparison to a naive noising approach shows that the proposed noising approach is particularly relevant. M OTS - CLÉS bruitage, reconnaissance automatique la parole, erreurs. K EYWORDS language understanding, data noising, recognition, errors.

This paper presents an approach to simulate automatic speech recognition (ASR) errors from manual transcriptions and describes how it can be used to improve the performance of spoken language understanding (SLU) systems. In particular, we point out that this noising process is very usefull to obtain a more robust SLU system to ASR errors in case of insufficient training data or more if ASR transcriptions are not available during the training of the SLU model. The proposed method is based on the use of both acoustic and linguistic word embeddings in order to define a similarity measure between words dedicated to predict ASR confusions. Actually, we assume that words acoustically and linguistically close are the ones confused by an ASR system. By using this similarity measure in order to randomly substitute correct words by potentially confusing words in manual annotations used to train CRF- or neural- based SLU systems, we augment the training corpus with these new noisy data. Experiments were carried on the French MEDIA corpus focusing on hotel reservation. They show that this approach significantly improves SLU system performance with a relative reduction of 21.2% of concept/value error rate (CVER), particularly when the SLU system is based on a neural approach (reduction of 22.4% of CVER). A comparison to a naive noising approach shows that the proposed noising approach is particularly relevant.

This paper addresses the problem of automatic speech recognition (ASR) error detection and their use for improving spoken language understanding (SLU) systems. In this study, the SLU task consists in automatically extracting, from ASR transcriptions , semantic concepts and concept/values pairs in a e.g touristic information system. An approach is proposed for enriching the set of semantic labels with error specific labels and by using a recently proposed neural approach based on word embeddings to compute well calibrated ASR confidence measures. Experimental results are reported showing that it is possible to decrease significantly the Concept/Value Error Rate with a state of the art system, outperforming previously published results performance on the same experimental data. It also shown that combining an SLU approach based on conditional random fields with a neural encoder/decoder attention based architecture , it is possible to effectively identifying confidence islands and uncertain semantic output segments useful for deciding appropriate error handling actions by the dialogue manager strategy .

L'etude porte sur l'apport d'un reseau de neurones recurrent (Recurrent Neural Network-RNN) bidirectionnel encodeur/decodeur avec mecanisme d'attention pour une tâche de compre-hension de la parole. Les premieres experiences faites sur le corpus ATIS confirment la qualite du systeme RNN etat de l'art utilise pour cet article, en comparant les resultats obtenus a ceux recemment publies dans la litterature. Des experiences supplementaires montrent que les RNNs avec mecanisme d'attention obtiennent de meilleures performances que les RNNs recemment proposes pour la tâche d'etiquetage en concepts semantiques. Sur le corpus MEDIA, un corpus francais etat de l'art pour la comprehension dedie a la reservation d'hotel et aux informations touristiques, les experiences montrent qu'un RNN bidirectionnel atteint une f-mesure de 79,51 tandis que le meme systeme integrant le mecanisme d'attention permet d'atteindre une f-mesure de 80,27

L’etude porte sur l’apport d’un reseau de neurones recurrent (Recurrent Neural Network RNN) bidirectionnel encodeur/decodeur avec mecanisme d’attention pour une tâche de comprehension de la parole. Les premieres experiences faites sur le corpus ATIS confirment la qualite du systeme RNN etat de l’art utilise pour cet article, en comparant les resultats obtenus a ceux recemment publies dans la litterature. Des experiences supplementaires montrent que les RNNs avec mecanisme d’attention obtiennent de meilleures performances que les RNNs recemment proposes pour la tâche d’etiquetage en concepts semantiques. Sur le corpus MEDIA, un corpus francais etat de l’art pour la comprehension dedie a la reservation d’hotel et aux informations touristiques, les experiences montrent qu’un RNN bidirectionnel atteint une f-mesure de 79,51 tandis que le meme systeme integrant le mecanisme d’attention permet d’atteindre une f-mesure de 80,27.

This study explores the use of a bidirectional recurrent neural network (RNN) encoder/decoder based on a mechanism of attention for a Spoken Language Understanding (SLU) task. First experiments carried on the ATIS corpus confirm the quality of the RNN baseline system used in this paper, by comparing its results on the ATIS corpus to the results recently published in the literature. Additional experiments show that RNN based on a mechanism of attention performs better than RNN architectures recently proposed for a slot filling task. On the French MEDIA corpus, a French state-of-the-art corpus for SLU dedicated to hotel reservation and tourist information, experiments show that a bidirectionnal RNN reaches a f-measure value of 79.51 while the use of a mechanism of attention allows us to reach a f-measure value of 80.27.