Validation of an Automated Scoring Program for a digital Complex Figure Copy Task within healthy ageing and stroke

ObjectiveComplex Figure Copy Tasks are one of the most commonly employed neuropsychological tests. However, manual scoring of this test is time-consuming, requires training, and can then still be inconsistent between different examiners. We aimed to develop and evaluate a novel, automated method for scoring a tablet-based Figure Copy Task. MethodA cohort of 261 healthy adults and 203 stroke survivors completed the computerised Oxford Cognitive Screen – Plus Figure Copy Task. Responses were independently scored by two trained human raters and by a novel automated scoring program. ResultsOverall, the Automated Scoring Program was able to reliably extract and identify the separate figure elements (average sensitivity and specificity of 92.10% and 90.20% respectively) and assigned total scores which agreed well with manual scores (ICC = .83). Receiver Operating Curve analysis demonstrated that, compared to overall impairment categorisations based on manual scores, the Automated Scoring Program had an overall sensitivity and specificity of 80% and 93.40% respectively (AUC = 86.70%). Automated total scores also reliably distinguished between different clinical impairment groups with acute stroke survivors scoring significantly worse than longer term survivors, which in turn scored worse than neurologically healthy adults. ConclusionsThese results demonstrate that the novel automated scoring algorithm was able to reliably extract and accurately score Figure Copy Task data, even in cases where drawings were highly distorted due to comorbid fine-motor deficits. This represents a significant advancement as this novel technology can be employed to produce immediate, unbiased, and reproducible scores for Figure Copy Task responses in clinical and research environments.