Dataset 2: Recommended Language Models

These word models were trained with a sentence start word of< s>, a sentence end word of, and an unknown word< unk>. The word vocabulary was the most frequent 64K words in the forum dataset that were also in a list of 330K known English words. All words are in lowercase. The character models are 12-gram models and were trained using interpolated Witten-Bell smoothing. The character model vocabulary consists of the lowercase letters az, apostrophe,< sp>; for a space,< s> for sentence start, and for sentence end.