Pronunciation assessment remains a subjective task which depends on a pronunciation reference hold as canonical. Whether a second language (L2) speaker is able to replicate said reference is decided by an assessor who perceives the identity of the sounds produced. It is known that the assessor has a bias caused by the perception of the speaker, hence the definition of a standard for L2 pronunciation is crucial in a formal assessment. In Computer Assisted Pronunciation Assessment (CAPA), the definition of a pronunciation standard for L2 is not trivial due to limited L2 data annotated for mispronunciations. Inspired on the assessor’s bias, this work explores an alternative to a conventional Automatic Speech Recognition approach for CAPA by using speaker metadata along with acoustic observations for mispronunciation detection. A combination of Bidirectional Long-Short Memory with self-attention was used to detect pronunciation errors in short speech segments. It was found that the use of categorical metadata can have a positive effect in the classification of mispronounced segments depending on the sparsity and balance of the classes. It was also found that different assessors can be influenced differently by information about the speaker’s linguistic background. The effect of the metadata was tested on data from Dutch children learners of English as L2 in schools across the Netherlands. The limited speaker diversity of the corpus made the task a challenge worth keep exploring.
Algorithmic journalism refers to automatic AI-constructed news stories. There have been successful commercial implementations for news stories in sports, weather, financial reporting and similar domains with highly structured, well defined tabular data sources. Other domains such as local reporting have not seen adoption of algorithmic journalism, and thus no automated reporting systems are available in these categories which can have important implications for the industry. In this paper, we demonstrate a novel approach for producing news stories on government legislative activity, an area that has not widely adopted algorithmic journalism. Our data source is state legislative proceedings, primarily the transcribed speeches and dialogue from floor sessions and committee hearings in US State legislatures. Specifically, we create a library of potential events called phenoms. We systematically analyze the transcripts for the presence of phenoms using a custom partial order planner. Each phenom, if present, contributes some natural language text to the generated article: either stating facts, quoting individuals or summarizing some aspect of the discussion. We evaluate two randomly chosen articles with a user study on Amazon Mechanical Turk with mostly Likert scale questions. Our results indicate a high degree of achievement for accuracy of facts and readability of final content with 13 of 22 users in the first article and 19 of 20 subjects of the second article agreeing or strongly agreeing that the articles included the most important facts of the hearings. Other results strengthen this finding in terms of accuracy, focus and writing quality.
While research of sentiment analysis became very popular on the global scope, in Slovak language as an under-resourced language there are still many issues to be tackled, especially the lack of resources. In this paper, we introduce a sentiment analysis game designed to collect sentiment annotations. The game is intended for a single player who, motivated by game score, chooses the sentiment category of each word of a sentence. We describe the annotation collection process during which over 12 500 annotations of individual words and over 1 000 annotations of entire sentences were obtained within a week. The collected annotations were used to construct a sentiment lexicon. Using artificial bee colony algorithm, optimal lexicon construction parameters were discovered. To evaluate the final lexicon's usefulness, we applied simple sentence-level lexicon-based sentiment analysis methods on a manually annotated dataset of mobile phone reviews. The same was done with other existing lexicons for comparison. The results of our experiments show that collecting annotations using our game can be useful as a method for constructing a sentiment lexicon.
The emergence of smart home assistants increased the need for robust Far-Field Speaker Identification models. Speaker Identification enables the assistants to perform personalized tasks. Smart home assistants face very challenging speech conditions, including various room shapes and sizes, various distances of the speaker from the microphone, various types of distractor noises (TV in the background, air conditioner, fridge, babble speech of other speakers, etc.). This paper describes the use of Invariant Representation Learning (IRL) as a method aimed to increase the robustness of Speaker Identification models on Far-Field. We introduce three new versions of IRL: Text-Dependent IRL (TD-IRL), Text Independent IRL (TI-IRL), and Deep Features IRL (DF-IRL). We evaluate the IRL models performance and compare them to the base x-vector model. The various Far-Field scenarios are evaluated using VOiCES dataset - a dataset of simulated Far-Field recordings in four real furnished rooms. TD-IRL and DF-IRL improve the minDCF results on the far-field scenarios by an average of 36%, and TI-IRL improves it by 31% with respect to the baseline model.
Recent developments in Named Entity Recognition (NER) have demonstrated good results for grammatically correct texts, even in low resourced settings. However, when the NER model faces ungrammatical text, it often shows poor performance. In this study, we analyze NER performance on datasets containing errors typical for user-generated texts in the Latvian language. We explore three different strategies to increase the robustness of the named entity recognition: error injection into grammatically correct texts, augmenting grammatically correct texts with erroneous texts and augmenting grammatically correct texts with erroneous texts that contain specific types of errors. We demonstrate that in low resourced settings, the best noise-robust model could be obtained by augmenting training data with datasets containing different error types. Our best model achieves an average F1 score of 83.5 (84.1 for baseline) on grammatically correct text, while keeping good performance (79 F1 vs. 66 for baseline) on noisy texts.
