Intelligent Patient Assessment & Monitoring System: Developing an Emotion Recognition Algorithm for Vocal Cues

Research Objectives This project will develop an intelligent Patient Assessment & Monitoring System (iPAMS) consisting of different sensor networks and smart software algorithms with the ultimate goal to monitor and assess patients with TBI in the rehabilitation setting. Design Cross-sectional study. Setting The study took place at the Centre for Applied Neuroscience, University of Cyprus. Participants The algorithm developed was tested on a set of data collected from 24 neurotypical participants (male = 8; female = 16), with an age range of 18-51 (M = 27.08; SD = 7.34). The data set of the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) was used to create the algorithm of the Online Speech Emotion Recognition using Python 3.7.6. The code of speech expressions included happy, sad, angry, fearful, disgust, surprise, and neutral. The gender differentiation was also inserted to detect the different emotions between men and women. The output is a real-time speech emotion recognition. Interventions Not applicable. Main Outcome Measures A sophisticated online algorithm has been developed to detect and classify vocal signals into different emotions. Results The speech recognition algorithm developed and implemented was shown to effectively detect and classify all targeted emotions, in men and women. Most importantly, this speech recognition technique can be applied in real-time emotion detection. Conclusions These findings suggest that real-time voice processing algorithms could contribute towards the understanding of the emotional state of patients, more efficiently, prior to the presence of any disruptive behavior. The idea behind this is that this software algorithm could specifically identify a potential behavioral alert through a personalized patient profile and notify the rehabilitation facility and the patient with TBI with real-time feedback. Author(s) Disclosures None disclosed.