Graph Edit Distance for the analysis of children's on-line handwritten arithmetical operations

This paper is based on a research project aiming at improving learning arithmetic operations at school using pen-based tablets. Given an arithmetic operation instruction, the goal is to analyze a child's handwritten answer. This comes down to find if any mistakes are made and their nature. An adapted representation and similarity search are needed for this analysis. In this paper, we propose to use a valued graph representation for handwritten arithmetical operations. To produce the analysis, we compute a similarity search with the corresponding expected answer using Graph Edit Distance (GED). To make up for the uncertainty of the noisy handwritten input recognition, we produce several segmented graph hypotheses for a single answer. Using the GED, we are able to correlate each hypothesis to the instruction graph. It enables to highlight multiple kinds of mistakes a child can make. The GED computation being a NP-complete problem, we propose to use sub-graph isomorphism: we partially match the instruction on each hypothesis in polynomial time to cut part of the tree search. Experiments were conducted on an in-house dataset composed of 400 handwritten arithmetical additions written by children on pen-based tablet. The time required for the GED computation is evaluated. We are able to match the complete operation in reasonable time on larger graphs while finding most of the time the best corresponding hypothesis.