A Hierarchical Bayesian Model for Cyber-Human Assessment of Rehabilitation Movement

Background The evidence-based quantification of the relation between changes in movement quality and functionality can assist clinicians in achieving more effective structuring or adaptations of therapy. Facilitating this quantification through computational tools can also result in the generation of large-scale data sets that can inform automated assessment of rehabilitation. Interpretable automated assessment can leave more time for clinicians to focus on treatment and allow for remotely supervised therapy at the home. Methods In our first experiment, we developed a rating process and accompanying computational tool to assist clinicians in following a standardized movement assessment process relating functionality to movement quality. We conducted three studies with three different versions of the computational rating tool. Clinicians rated task, segment, and movement feature performance for 440 videos in which stroke survivors executed standardized upper extremity therapy tasks related to functional activities. In our second experiment, we used the 440 rated videos, in addition to 140 videos of unimpaired subjects performing the same tasks, to improve our previously developed automated assessment ensemble model that automatically generates segmentation times and task ratings across impaired and unimpaired movement. The automated assessment ensemble integrates expert knowledge constraints into data driven training though a combination of HMM, transformer, MSTCN++, and decision tree computational modules. In our third experiment, we used the therapist and automated ratings to develop a four-layer Hierarchical Bayesian Model (HBM) for computing the statistical relation of movement quality changes to functionality. We first calculated conditional layer probabilities using clinician ratings of task, segment, and movement features. We increased the granularity of observation of the HBM by formulating Δ HBM , a correlation graph between kinematics and movement composite features. Finally, we used k-means clustering on the Δ HBM to identify three clusters of features among the 16 movement composite and 20 kinematic features and used the centroid of these clusters as the weights of the input data to our computational assessment ensemble. Results We evaluated the efficacy of our rating interface in terms of inter-rater reliability (IRR) across tasks, segments, and movement features. The third version of the interface produced an average IRR of 67%, while the time per session (TPS) was the lowest of the three studies. By analyzing the ratings, we were able to identify a small number of movement features that have the highest probability of predicting functional improvement. We evaluated the performance of our automated assessment model using 60% impaired and 40% unimpaired movement data and achieved a frame-wise segmentation accuracy of 87.85±0.58 and a block-segmentation accuracy of 98.46±1.6. We also demonstrated the performance of our proposed HBM in correlation to clinician’s ratings with a correlation over 90%. The HBM also generates a correlation graph, Δ HBM that relates 16 composite movement features to the 20 kinematic features. We can thus integrate the HBM into the computational assessment ensemble to perform automated and integrated movement quality and functionality assessment that is driven by computationally extracted kinematics. Conclusions Combining standardized clinician ratings of videos with knowledge based and data driven computational analysis of rehabilitation movement allows the expression of an HBM that increases the observability of the relation of movement quality to functionality and enables the training of computational algorithms for automated assessment of rehabilitation movement. While our work primarily focuses on the upper extremity of stroke survivors, the models can be adopted to many other neurorehabilitation contexts. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This material is based upon work supported by the National Science Foundation under Grant No. (2014499) and the National Institute on Disability, Independent Living, and Rehabilitation Research under Award No. 90REGE0010. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Virginia Tech IRB IRB #18-877 Evaluating Interactive Systems for the Assessment of Movement Quality in Healthy Control Participants Virginia Tech IRB IRB #19-920 Understanding the activities and documentation needs of occupational and physical therapists Virginia Tech IRB IRB #17-379 Computational assessment of upper extremity movement from video data Emory University IRB IRB #97319 Evaluating interactive systems for semi-automated assessment of movement functionality and movement quality of impaired subjects I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes All data produced in the present study are available upon reasonable request to the authors