Towards the Improvement of the Cognitive Process of the Synthesized Speech of Mathematical Expression in MathML: An Eye-Tracking

Being one essential part of the solutions we are developing to provide accessibility for blind persons, synthesized speech of mathematical content, although having evolved in naturalness in recent years, still keeps a characteristic of generating very long outputs, which makes it difficult for a person who uses a screen reader to memorize. Eye-gaze tracking has been useful for studying cognitive processes and student's effort in performing tasks therefore it seems attractive as a means to understand how people analyze, process, and break down extensive math expressions into smaller ones. This understanding can help to build a similar model of auditory rendering of MathML capable of avoiding cognitive overload. Therefore, we carried out an experiment to do a gaze-based analysis of reading and memorizing mathematical expressions. Our sample consisted of fourteen participants, and seven mathematical expressions were used in the experiment. The results showed that the computed complexity of the expressions and the average duration of eye-gaze fixations have a strong and significant relationship. Therefore, we conclude that the complexity variable must always be considered in the speech synthesis process of mathematical expressions. However, when considering why participants requested the repetition of an expression with not larger complexity during the memorization process, a tentative interpretation based on the number of MathML tree nodes and the computed complexity of the expressions was not enough to explain it, indicating that other factors need to be investigated. Although some answers to the research questions were not yet conclusive, we consider this research relevant because it points to directions for the work in order to minimize the cognitive processing and mental load of the synthesized speech rendering of mathematical expressions.