Addressing Accent Mismatch In Mandarin-English Code-Switching Speech Recognition

Automatic speech recognition systems suffer from accuracy degradation when code-switching (multiple languages are spoken in a single utterance) is encountered. This is especially common for non-native speakers where there is a mismatch between speech and acoustic model. In this paper, we experiment on Mandarin-English code-switching audio spoken by native Chinese speakers and evaluate three techniques to improve accuracy-data adaptation, individual senone modeling and lexicon enrichment. Our results show the recognition of accented speech improves up to 12% on various code-switching datasets. We also propose several metrics to measure code-switching recognition quality, not captured in typical word error rate (WER) measurement.