Handwritten Essay Grading On Mobiles Using Mdlstm Model And Word Embeddings

Assessing handwritten essays is a human skill which is very important for school level language exams. If automated, it will enable scalable assessment and feedback at low cost. This problem involves two modalities, viz. images for Offline Handwriting Recognition (OHR) and Natural Language Processing (NLP) for essay grading. We consider the sequential information of handwriting for getting the transcriptions from text images. We train a Multidimensional Long Short Term Memory (MDLSTM) network with Connectionist Temporal Classification (CTC) cost function at the output for the task of OHR. The paper discusses the generalization of the handwriting recognition model for images taken from scanner and mobile camera. Further a comparison of results of essay grading is shown for features of essays based on GloVe and fastText based word vector representation models. We trained different models for the essay grading task considering it both as a classification and regression problem. The results show that the state of the art fastText word vector representation based features for essays perform better than the other features considered in this work. The best performing model shows Quadratic Weighted Kappa (QWK) agreement of 0.80 for grading between the human graded text essays and model graded text essays. The same model shows the QWK agreement of 0.81 for grading between the human graded text essays and the OHR transcribed essays. In this work, we consider handwritten essays written in English.