Investigating End-To-End Speech Recognition For Mandarin-English Code-Switching

Code-switching is a common phenomenon in many multilingual communities and presents a challenge to automatic speech recognition ( ASR). In this paper, three approaches are investigated to improve end-to-end speech recognition on Mandarin-English code-switching task. First, multi-task learning ( MTL) is introduced which enables the language identity information to facilitate Mandarin-English code-switching ASR. Second, we explore wordpieces, as opposed to graphemes, as English modeling units to reduce the modeling unit gap between Mandarin and English. Third, we employ transfer learning to utilize larger amount of monolingual Mandarin and English data to compensate the data sparsity issue of a code-switching task. Significant improvements are observed from all three approaches. With all three approaches combined, the final system achieves a character error rate ( CER) of 6.49% on a real Mandarin-English code-switching task.