Comparing Open-Source Speech Recognition Toolkits ⋆

In this paper, a large-scale evaluation of open-source speech recognition toolkits is described. Specifically, HTK in association with the decoders HDecode and Julius, CMU Sphinx with the decoders pocketsphinx and Sphinx-4, and the Kaldi toolkit are compared in terms of usability and expense of recognition accuracy. The evaluation presented in this paper was done on German and English language using respective the Verbmobil 1 and the Wall Street Journal 1 corpus. We found that, Kaldi providing the most advanced training recipes gives outstanding results out of the box, the Sphinx toolkit also containing recipes enables good results in short time. HTK provides the least support and requires much more time, knowledge and effort to obtain results of the order of the other toolkits.