Reading acquisition is one the main keys for school success and a crucial component for empowering individuals to participate meaningfully in society. Yet, it is still a challenging skill to acquire for around 10% of children that have dyslexia, a type of neuro-developmental disorder that affects the ability to learn how to read and write. Dyslexia is often under-diagnosed, and normally children with dyslexia are only detected once they fail in school, even though dyslexia is not related to general intelligence. In this work, we present an approach for screening dyslexia using language-independent games in combination with machine learning models. To reach this goal, we designed the content of a computer game, collected data from 137 children playing this game (51 with dyslexia) in different languages -German, Spanish and English- and created a prediction model using different machine learning classifiers. Our method provides a precision of 0.78 and recall of 0.79 for German and a precision of 0.83 and recall of 0.80 for all languages when Extra Trees are used, with an accuracy of 0.67 and 0.75, respectively. Our results open the possibility of inexpensive online early screening of dyslexia for young children using non-linguistic elements.
In this paper, we evaluate the creative fiction writing abilities of a fine-tuned small language model (SLM), BART Large, and compare its performance to humans and two large language models (LLMs): GPT-3.5 and GPT-4o. Our evaluation consists of two experiments: (i) a human evaluation where readers assess the stories generated by the SLM compared to human-written stories, and (ii) a qualitative linguistic analysis comparing the textual characteristics of the stories generated by the different models. In the first experiment, we asked 68 participants to rate short stories generated by the models and humans along dimensions such as grammaticality, relevance, creativity, and attractiveness. BART Large outperformed human writers in most aspects, except creativity, with an overall score of 2.11 compared to 1.85 for human-written texts – a 14 revealed that, while GPT-4o exhibited near-perfect internal and external coherence, it tended to produce more predictable narratives, with only 3 its stories seen as novel. In contrast, 15 novel, indicating a higher degree of creativity despite its smaller model size. This study provides both quantitative and qualitative insights into how model size and fine-tuning influence the balance between creativity, fluency, and coherence in creative writing tasks.
We evaluate a reading intervention involving 600 third-grade students in Chilean schools catering to disadvantaged populations. The intervention features an adaptive computer game designed to identify and improve weaknesses in literacy and cognitive skills, and is complemented by a mobile library and advice to parents to increase student's interest and parental involvement. We first quantify the impact on non-cognitive skills and academic perceptions. We find that, after just three months of intervention, treated students are 2030 percent of a standard deviation more likely to believe that their performance is better than that of their peers, to like school, to have stronger grit, and to have a more internal locus-of-control. Gains in aspirations and self-confidence are particularly large for students that we identify as at-risk-of-dyslexia. These improvements are reflected in better performance on a nation-wide, standardized language test. Our results show that non-cognitive skills, particularly of at-risk-of-dyslexia students, can be changed through a short, light-touch, and cost-effective education technology intervention.
Children with dyslexia have difficulties learning how to read and write. They are often diagnosed after they fail school even if dyslexia is not related to general intelligence. Early screening of dyslexia can prevent the negative side effects of late detection and enables early intervention. In this context, we present an approach for universal screening of dyslexia using machine learning models with data gathered from a web-based language-independent game. We designed the game content taking into consideration the analysis of mistakes of people with dyslexia in different languages and other parameters related to dyslexia like auditory perception as well as visual perception. We did a user study with 313 children (116 with dyslexia) and train predictive machine learning models with the collected data. Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as a F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. We also present the game content design, potential new auditory input, and knowledge about the design approach for future research to explore Universal screening of dyslexia. universal screening with language-independent content can be used for the screening of pre-readers who do not have any language skills, facilitating a potential early intervention.
This paper presents the research and the entrepreneurial journey behind Dytective. Dytective is a tool that combines machine learning and computer games to detect risk of dyslexia and ameliorate the symptoms of dyslexia through personalized exercises. It has been used over 325,000 times, becoming the most used dyslexia online screener for Spanish. Recently, this platform has been adopted by over 800 Spanish public schools in collaboration with Regional Governments.
We present the results of a study that tests the creative writing abilities of Transformers (current state-of-the-art Deep Neural Networks for Natural Language Processing) with respect to humans. In our experiment, transformers are given a title, and their task is to invent a synopsis for a movie that matches the title. We collected 24,480 manual assessments on synopsis written by transformers and humans that, altogether, reveal that the synopsis generated by transformers are, in average, significantly better than their human counterparts in terms of readability, understandability, relevance with respect to the title, informativity and attractiveness. The only aspect where transformers match, but do not improve, human performance is creativity. Our results also indicate that, if assessors are informed of who is the author of the text (human or machine), machine-made synopsis receive lower scores, confirming that the interpretation of a creative text depends on the expectations of the reader with respect to the author. This is, to our knowledge, the first experiment on creative writing where transformers show true superhuman performance. Our result confirms the potential of transformers to assist creative writers, but also calls for a reflection on the methodological limitations and challenges of evaluating creative tasks.
This paper presents the research and the entrepreneurial journey behind Dytective. Dytective is a tool that combines machine learning and computer games to detect risk of dyslexia and ameliorate the symptoms of dyslexia through personalized exercises. It has been used over 325,000 times, becoming the most used dyslexia online screener for Spanish. Recently, this platform has been adopted by over 800 Spanish public schools in collaboration with Regional Governments.
Background: The use of electronic interventions to improve reading is becoming a common resource. This systematic review aims to describe the main characteristics of randomized controlled trials or quasi-experimental studies that have used these tools to improve first-language reading, in order to highlight the features of the most reliable studies and guide future research.Methods: The whole procedure followed the PRISMA guidelines, and the protocol was registered before starting the process (doi: 10.17605/OSF.IO/CKM4N). Searches in Scopus, PubMed, Web of Science and an institutional reference aggregator (Unika) yielded 6,230 candidate articles. After duplicate removal, screening, and compliance of eligibility criteria, 55 studies were finally included.Results: They were research studies on improving first-language reading, both in children and adults, and including a control group. Thirty-three different electronic tools were employed, most of them in English, and studies were very diverse in sample size, length of intervention, and control tasks. Risk of bias was analyzed with the PEDro scale, and all studies had a medium or low risk. However, risk of bias due to conflicts of interest could not be evaluated in most studies, since they did not include a statement on this issue.Conclusion: Future research on this topic should include randomized intervention and control groups, with sample sizes over 65 per group, interventions longer than 15 h, and a proper disclosure of possible conflicts of interest.Systematic Review Registration: The whole procedure followed the PRISMA guidelines, and the protocol was registered before starting the process in the Open Science Framework (doi: 10.17605/OSF.IO/CKM4N).
En este artículo evaluamos el programa de Ayuda a la Dislexia de la Comunidad de Madrid en el curso 2018-2019 dirigido a alumnos de primaria con dificultades lectoescritoras. El programa consiste en dos herramientas: a) una prueba para comprobar los alumnos que presentan dificultades de lectoescritura, y b) hasta 42.000 ejercicios en una plataforma digital para trabajar y reforzar esas competencias. Un 7,55 % (n=1,022) de los alumnos presentaban dificultades lectoescritoras. Los resultados del análisis muestran que no existe correlación entre la participación del centro en el programa y el rendimiento de los alumnos de esos colegios en las pruebas externas y estandarizadas de Matemáticas y Lengua, pero sí para el caso de Inglés. Al desglosar los resultados por género, se observa una asociación fuerte y positiva del proyecto Ayuda a la Dislexia para las chicas en Lengua e Inglés (y, como se podría esperar, no tanto en Matemáticas), mientras que para los chicos solo es significativa en Inglés. No obstante, se ha comprobado que los centros que participan en esta intervención tenían mejores resultados que los centros no tratados y que sus alumnos provienen de un entorno socioeconómico más elevado. Las diferentes características de los colegios tratados podrían explicar su participación, que era voluntaria, en el programa de Ayuda a la Dislexia. La línea de investigación futura plantea explotar a través de un análisis de diferencias-en-diferencias que ha habido una nueva convocatoria del programa en el curso 2021-2022, en la que se han incorporado nuevos centros educativos para analizar si existe causalidad.
Dyslexia is a specific learning disorder related to school failure. Detection is both crucial and challenging, especially in languages with transparent orthographies, such as Spanish. To make detecting dyslexia easier, we designed an online gamified test and a predictive machine learning model. In a study with more than 3,600 participants, our model correctly detected over 80% of the participants with dyslexia. To check the robustness of the method we tested our method using a new data set with over 1,300 participants with age customized tests in a different environment -a tablet instead of a desktop computer- reaching a recall of over 78% for the class with dyslexia for children 12 years old or older. Our work shows that dyslexia can be screened using a machine learning approach. An online screening tool in Spanish based on our methods has already been used by more than 200,000 people.
Children with dyslexia are often diagnosed after they fail school even if dyslexia is not related to general intelligence. In this work, we present an approach for universal screening of dyslexia using machine learning models with data gathered from a web-based language-independent game. We designed the game content taking into consideration the analysis of mistakes of people with dyslexia in different languages and other parameters related to dyslexia like auditory perception as well as visual perception. We did a user study with 313 children (116 with dyslexia) and train predictive machine learning models with the collected data. Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as a F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. To the best of our knowledge this is the first time that risk of dyslexia is screened using a language-independent content web-based game and machine-learning. Universal screening with language-independent content can be used for the screening of pre-readers who do not have any language skills, facilitating a potential early intervention.
Detecting dyslexia is important because early intervention is key to avoid the negative effects of dyslexia such as school failure. Most of the current approaches to detect dyslexia require expensive personnel (i.e. psychologists) or special hardware (i.e. eye trackers or MRI machines). Also, most of the methods can only be used when children are learning how to read but not before, necessarily delaying needed early intervention. In this work, we present a study with 178 participants speaking different languages (Spanish, German, English, and Catalan) with and without dyslexia using a web-based game built with musical and visual elements that are language independent. The study reveals eighth game measures with significant differences for Spanish children with and without dyslexia, which could be used in future work as a basis for language independent detection. A web-based application like this could have a major impact on children all over the world by easily screening them and suggest the help they need.
This demo describes an ongoing research project that aims to develop a video game for the training of two independent cognitive components involved in reading development: visual attention and auditory rhythm. The video game includes two types of gaming activities for each component. First, a proof of concept was carried out with 10 children with dyslexia. The outcome of this proof of concept study served as foundation for the development of a prototype that has been assessed. Human-computer interaction, usability and engagement were measured in a user study with 22 children with dyslexia and 22 without dyslexia. Significant interaction differences between group were not found. Usability and engagement evaluation was positive and will be used to improve the video game. Its efficacy will be tested with a longitudinal training study in developing readers. A video of Jellys user testing is available in https://youtu.be/T9oO9bZFdmM.
Using serious games to screen dyslexia has been a successful approach for English, German and Spanish. In a pilot study with a desktop game, we addressed pre-readers screening, that is, younger children who have not acquired reading or writing skills. Based on our results, we have redesigned the game content and new interactions with visual and musical cues. Hence, here we present a tablet game, DGames , which has the potential to predict dyslexia in pre-readers. This could contribute to around 10% of the population that is affected by dyslexia, as children will gain more time to learn to cope with the challenges of learning how to read and write.