In this paper, we extend our previous work on the detection of system-directed speech utterances. This type of binary classification can be used by virtual assistants to create a more natural and fluid interaction between the system and the user. We explore two methods that both improve the Equal-Error-Rate (EER) performance of the previous model. The first exploits the supplementary information independently captured by ASR models through integrating ASR decoder-based features as additional inputs to the final classification stage of the model. This relatively improves EER performance by 13%. The second proposed method further integrates word embeddings into the architecture and, when combined with the first method, achieves a significant EER performance improvement of 48%, relative to that of the baseline.

Estimating the quality of speech without the use of a clean reference signal is a challenging problem, in part due to the time and expense required to collect sufficient training data for modern machine learning algorithms. We present a novel, non-intrusive estimator that exploits recurrent neural network architectures to predict the intrusive POLQA score of a speech signal in a short time context. The predictor is based on a novel compressed representation of modulation domain features, used in conjunction with static MFCC features. We show that the proposed method can reliably predict POLQA with a 300 ms context, achieving a mean absolute error of 0.21 on unseen data. The proposed method is trained using English speech and is shown to generalize well across unseen languages. The neural network also jointly estimates the mean voice activity detection (VAD) with an F1 accuracy score of 0.9, removing the need for an external VAD.

Speaker diarization in the meeting scenario is a challenging problem due to spontaneous, conversational speech and distant nature of signal acquisition. We present a novel method that operates with the distant microphones placed in unknown locations and with ad-hoc configurations in a room. The method exploits robust spatial features and combines them with noise robust spectral features, enhanced using a modulation domain algorithm. The spatial features extracted include a model of the time-difference-of-arrival (TDOA) information along with optimal channel-pair selection, based on which enhanced spectral features are computed. We present results on the NIST RT05 meeting corpora and show that the proposed method significantly reduces the diarization error rate compared to a baseline diarization system, outperforming a previous method by a relative 33.1% DER.

In studies on artificial bandwidth extension (ABE), there is a lack of international coordination in subjective tests between multiple methods and languages. Here we present the design of absolute category rating listening tests evaluating 12 ABE variants of six approaches in multiple languages, namely in American English, Chinese, German, and Korean. Since the number of ABE variants caused a higher-than-recommended length of the listening test, ABE variants were distributed into two separate listening tests per language. The paper focuses on the listening test design, which aimed at merging the subjective scores of both tests and thus allows for a joint analysis of all ABE variants under test at once. A language-dependent analysis, evaluating ABE variants in the context of the underlying coded narrowband speech condition showed statistical significant improvement in English, German, and Korean for some ABE solutions.

In this paper, we extend our previous work on exploiting speech temporal properties to improve Bandwidth Extension (BWE) of narrowband speech using Gaussian Mixture Models (GMMs). By quantifying temporal properties through information theoretic measures and using delta features, we have shown that narrowband memory significantly increases certainty about highband parameters. However, as delta features are non-invertible, they can not be directly used to reconstruct highband frequency content. In the work presented herein, we embed temporal properties indirectly into the GMM structure through a memorydependent tree-based approach to extend representation of the narrow band. In particular, sequences of past frames are progressively used to grow the GMM in a tree-like fashion. This growth approach results in reliable estimates for the GMM parameters such that Maximum Likelihood estimation is no longer necessary, thus circumventing the complexity accompanying high-dimensionality GMM training.

In this paper, we continue our previous work on improving Bandwidth Extension (BWE) of narrowband speech. We have shown that including memory into the parametrization frontend (through delta features) results in higher highband certainty irrespective of feature type, with MFCCs exhibiting higher correlation, in general, between both bands, reaching twice that using LSFs. By incorporating memory into the frontend of a conventional LP-based BWE system, we were able to translate the higher correlation due to memory into BWE performance improvement. Using high-resolution inverse DCT, we also achieved high quality speech reconstruction from MFCCs, thus enabling MFCC-based BWE with improved performance compared to conventional static LP-based BWE. We continue this work by incorporating the superior correlation properties of frontend memory into our MFCC-based BWE system. Log-Spectral Distortion as well as the more perceptually-correlated Itakura-based measures show that incorporating memory into our MFCC-based BWE system results in BWE performance superior to that of our dynamic LP-based BWE system.

We present a novel MFCC-based scheme for the Bandwidth Extension (BWE) of narrowband speech. BWE is based on the assumption that narrowband speech (0.3-3.4 kHz) correlates closely with the highband signal (3.4-7 kHz), enabling estimation of the highband frequency content given the narrow band. While BWE schemes have traditionally used LP-based parametrizations, our recent work has shown that MFCC parametrization results in higher correlation between both bands reaching twice that using LSFs. By employing high-resolution IDCT of highband MFCCs obtained from narrowband MFCCs by statistical estimation, we achieve high-quality highband power spectra from which the time-domain speech signal can be reconstructed. Implementing this scheme for BWE translates the higher correlation advantage of MFCCs into BWE performance superior to that obtained using LSFs, as shown by improvements in log-spectral distortion as well as Itakura-based measures (the latter improving by up to 13%).

For the purpose of improving Bandwidth Extension (BWE) of narrowband speech, we continue our recent work on the positive effect of exploiting the temporal correlation of speech on the dependence between speech frequency bands. We have shown that such memory inclusion into MFCC speech parametrization translates into higher highband certainty. In the work presented herein, we employ VQ to estimate highband discrete entropies, thus refining our analysis of the effect of memory inclusion on increasing highband certainty. Moreover, we extend our previous analysis to LSF parameters. We further construct a BWE system that exploits our memory inclusion technique, thus translating highband certainty gains into practical BWE performance improvement as measured by the objective quality of reconstructed speech. Results show that memory inclusion decreases the log-Spectral Distortion of the extended highband speech by as much as 1 dB corresponding to more than 14% relative. Index Terms: Bandwidth Extension, Mutual Information.

In this paper, we investigate the effect of temporal correlation on the dependence between the speech narrow and high frequency bands covering the 0.3-3.4 kHz and 3.7-8 kHz ranges, respectively. We follow the technique of using Gaussian mixture modelling of spectral envelopes represented by Mel-frequency cepstral coefficients. The correlation between the disjoint speech frequency bands is quantified through mutual information (MI) and its ratio to highband entropy. Speech exhibits considerable temporal correlation that is not explicitly accounted for by static parametrization of spectral envelopes. Including memory in speech parametrization (through delta features) incorporates such temporal information of speech in its modelling, and hence, MI gains are to be expected resulting in bandwidth extension with better performance. Results show that exploiting delta features can increase certainty about the highband (ratio of MI to highband entropy) by as much as 216% relatively, corresponding to an absolute increase of 12%

Packet voice communications generally suffer packet losses as a result of various network- or transmission-related impairments. Upon decoding, these lost packets result in missing speech segments that degrade automatic speech recognition (ASR) performance. We present a novel loss recovery scheme that reproduces the missing speech waveform by interpolating its spectrum from the speech spectra on both sides of a loss. An adaptive mechanism is used to determine the FFT width of the speech waveform before and after a loss to capture as much spectral detail as possible. A linearly weighted spectral interpolation ensues to obtain the spectra of missing speech. The missing speech waveform is then reconstructed through IFFT, followed by smoothing at packet boundaries. Tests on Bluetooth voice packets with a high loss rate of 38% show that our scheme improves ASR performance considerably (up to 20%) while being computationally efficient, as it is an FFT-based scheme.

We investigate the ASR performance of speech transmitted over a noisy Bluetooth RF channel. Bluetooth shares its transmission channel with IEEE 802.11-based devices. Despite Bluetooth's frequency hopping scheme, our investigation shows that Bluetooth packet loss rates may reach up to 38% in unfavorable 802.11 interference conditions, and as Bluetooth uses a CVSD (continuous variable slope delta modulation) codec with syllabic companding, these packet losses not only manifest themselves as segments of missing speech upon CVSD decoding, but also as incorrect scaling of subsequent successfully received voice packets as CVSD step-size information is also lost. We investigate the effects of these degradations on the ASR performance of Bluetooth speech, and accordingly propose alternative CVSD decoder schemes employing insertion-based techniques for compensating for these effects. Results show that our proposed techniques improve ASR performance considerably while requiring only minor modifications to the current Bluetooth receiver.