Socially assistive robots are increasingly being explored to improve the engagement of older adults and people with disability in health and well-being-related exercises. However, even if people have various physical conditions, most prior work on social robot exercise coaching systems has utilized generic, predefined feedback. The deployment of these systems still remains a challenge. In this paper, we present our work of iteratively engaging therapists and post-stroke survivors to design, develop, and evaluate a social robot exercise coaching system for personalized rehabilitation. Through interviews with therapists, we designed how this system interacts with the user and then developed an interactive social robot exercise coaching system. This system integrates a neural network model with a rule-based model to automatically monitor and assess patients’ rehabilitation exercises and can be tuned with individual patient’s data to generate real-time, personalized corrective feedback for improvement. With the dataset of rehabilitation exercises from 15 post-stroke survivors, we demonstrated our system significantly improves its performance to assess patients’ exercises while tuning with held-out patient’s data. In addition, our real-world evaluation study showed that our system can adapt to new participants and achieved 0.81 average performance to assess their exercises, which is comparable to the experts’ agreement level. We further discuss the potential benefits and limitations of our system in practice.

Older adult care technologies are increasingly explored to support the independent living of older adults by monitoring their abnormal activities and informing caregivers to provide intervention if necessary. However, the adoption of these technologies remains challenging due to several factors (e.g. lack of usability). In this work, we present a human-centered, intelligent system for older adult care. Our proposed designs of the system were created based on the findings from a focus group session with caregivers. This system monitors the abnormal activities of an older adult using wireless motion sensors and machine learning models. In addition, unlike previous work that only notifies an outcome of activity recognition and abnormal detection models to a caregiver, the system supports interactive dialogue responses to explain the abnormal activities of an older adult to a caregiver and allow the caregiver to elicit additional information about the older adult and the older adult to proactively share his/her status with the caregiver for an adequate intervention.

BACKGROUND AND OBJECTIVES:Regular physical exercise can attenuate age-related cognitive decline. This study aimed to investigate the effect of a physical exercise multicomponent training based on exergames on cognitive functioning (CF) in older adults. RESEARCH DESIGN AND METHODS:This randomized controlled trial included older adults aged 61-78. Participants were randomly allocated to an intervention group (IG; n = 15) or active control group (CG; n = 16). The IG was exposed to a combined training with traditional exercise and exergaming, twice a week over a period of 12 weeks. The CG performed only traditional sessions. CF was assessed by the Cognitive Telephone Screening Instrument. The time points for assessment were at zero (pretest), 12 (posttest), and 17 weeks (follow-up). RESULTS:Active CG and IG increased from pretest to posttest in short-term memory (STM), long-term memory (LTM), and Cognitive Telephone Screening Instrument total score 1.98 > Z < 3.00, ps < .005, with moderately large positive effects (.36 > r < .54). A significant increase was seen from posttest to follow-up in STM, Z = 2.74, p = .006, and LTM, Z = 2.31, p < .021, only in IG. Across the two time periods posttest to follow-up, there were significant interaction effects between program type and time for STM (p = .022, ηp2=.17) and LTM (p = .004, ηp2=.25), demonstrating a more beneficial effect of the exergames intervention compared to the CG. Discussion and Implications: The integration of exergaming in a multicomponent functional fitness exercise might have the potential to maintain and improve CF (in particular, STM and LTM) in older adults.

Advances in artificial intelligence (AI) have made it increasingly applicable to supplement expert’s decision-making in the form of a decision support system on various tasks. For instance, an AI-based system can provide therapists quantitative analysis on patient’s status to improve practices of rehabilitation assessment. However, there is limited knowledge on the potential of these systems. In this paper, we present the development and evaluation of an interactive AI-based system that supports collaborative decision making with therapists for rehabilitation assessment. This system automatically identifies salient features of assessment to generate patient-specific analysis for therapists, and tunes with their feedback. In two evaluations with therapists, we found that our system supports therapists significantly higher agreement on assessment (0.71 average F1-score) than a traditional system without analysis (0.66 average F1-score, p < 0.05). After tuning with therapist’s feedback, our system significantly improves its performance from 0.8377 to 0.9116 average F1-scores (p < 0.01). This work discusses the potential of a human-AI collaborative system to support more accurate decision making while learning from each other’s strengths.

The research of a socially assistive robot has a potential to augment and assist physical therapy sessions for patients with neurological and musculoskeletal problems (e.g. stroke). During a physical therapy session, generating personalized feedback is critical to improve patient's engagement. However, prior work on socially assistive robotics for physical therapy has mainly utilized pre-defined corrective feedback even if patients have various physical and functional abilities. This paper presents an interactive approach of a socially assistive robot that can dynamically select kinematic features of assessment on individual patient's exercises to predict the quality of motion and provide patient-specific corrective feedback for personalized interaction of a robot exercise coach.

A robotic exercise coaching system requires the capability of automatically assessing a patient’s exercise to in-teract with a patient and generate corrective feedback. However, even if patients have various physical conditions, most prior work on robotic exercise coaching systems has utilized generic, pre-defined feedback.This paper presents an interactive approach that combines machine learning and rule-based models to automatically assess a patient’s rehabilitation exercise and tunes with patient’s data to generate personalized corrective feedback. To generate feedback when an erroneous motion occurs, our approach applies an ensemble voting method that leverages predictions from multiple frames for frame-level assessment. According to the evaluation with the dataset of three stroke rehabilitation exercises from 15 post-stroke subjects, our interactive approach with an ensemble voting method supports more accurate frame-level assessment (p < 0.01), but also can be tuned with held-out user’s unaffected motions to significantly improve the performance of assessment from 0.7447 to 0.8235 average F1-scores over all exercises (p < 0.01). This paper discusses the value of an interactive approach with an ensemble voting method for personalized interaction of a robotic exercise coaching system.

Cognitive assistants are computer-based interactive guides for complex activities such as emergency medical care. Recent advances in computing capability and edge computing make it possible to host such an assistant in a wearable device. We identify technical challenges facing the designer of a wearable cognitive assistant and describe pathways to solutions. We illustrate the solution space in the form of an automated end-to-end assistant to guide a novice through the application of an automatic external defibrillator (AED). Our results indicate that without such assistance, novices will fail, but technological barriers limit real-time success. Our work highlights challenges as well as new capabilities made possible by wearable technology in this domain. This work was supported by the National Science Foundation (NSF) under grant number CNS-

Clinical decision support systems have the potential to improve work flows of experts in practice (e.g. therapist's evidence-based rehabilitation assessment). However, the adoption of these systems is challenging, and the gains of these systems have not fully demonstrated yet. In this paper, we identified the needs of therapists to assess patient's functional abilities (e.g. alternative perspectives with quantitative information on patient's exercise motions). As a result, we co-designed and developed an intelligent decision support system that automatically identifies salient features of assessment using reinforcement learning to assess the quality of motion and generate patient-specific analysis. We evaluated this system with seven therapists using the dataset from 15 patients performing three exercises. The results show that therapists have higher usage intent on our system than a traditional system without patient-specific analysis ($p < 0.05$). While presenting richer information ($p < 0.10$), our system significantly reduces therapists' effort on assessment ($p < 0.10$) and improves their agreement on assessment from 0.66 to 0.71 F1-scores ($p < 0.01$). This work discusses the importance of human centered design and development of a machine learning-based decision support system that presents contextually relevant information and salient explanations on its prediction for better adoption in practice.

Technology-assisted systems to monitor and assess rehabilitation exercises have an opportunity of enhancing rehabilitation practices by automatically collecting patient's quantitative performance data. However, even if a complex algorithm (e.g. Neural Network) is applied, it is still challenging to develop such a system due to patients with various physical conditions. The system with a complex algorithm is limited to be a black-box system that cannot provide explanations on its predictions. To address these challenges, this paper presents a hybrid model that integrates a machine learning (ML) model with a rule-based (RB) model as an explainable artificial intelligence (AI) technique for quantitative assessment of stroke rehabilitation exercises. For evaluation, we collected therapist's knowledge on assessment as 15 rules from interviews with therapists and the dataset of three upper-limb stroke rehabilitation exercises from 15 post-stroke and 11 healthy subjects using a Kinect sensor. Experimental results show that a hybrid model can achieve comparable performance with a ML model using Neural Network, but also provide explanations on a model prediction with a RB model. The results indicate the potential of a hybrid model as an explainable AI technique to support the interpretation of a model and fine-tune a model with user-specific rules for personalization.

Automated assessment of rehabilitation exercises using machine learning has a potential to improve current rehabilitation practices. However, it is challenging to completely replicate therapist's decision making on the assessment of patients with various physical conditions. This paper describes an interactive machine learning approach that iteratively integrates a data-driven model with expert's knowledge to assess the quality of rehabilitation exercises. Among a large set of kinematic features of the exercise motions, our approach identifies the most salient features for assessment using reinforcement learning and generates a user-specific analysis to elicit feature relevance from a therapist for personalized rehabilitation assessment. While accommodating therapist's feedback on feature relevance, our approach can tune a generic assessment model into a personalized model. Specifically, our approach improves performance to predict assessment from 0.8279 to 0.9116 average F1-scores of three upper-limb rehabilitation exercises (p < 0.01). Our work demonstrates that machine learning models with feature selection can generate kinematic feature-based analysis as explanations on predictions of a model to elicit expert's knowledge of assessment, and how machine learning models can augment with expert's knowledge for personalized rehabilitation assessment.

Due to the limited number of therapists, task-oriented exercises are often prescribed for post-stroke survivors as in-home rehabilitation. During in-home rehabilitation, a patient may become unmotivated or confused to comply prescriptions without the feedback of a therapist. To address this challenge, this paper proposes an automated method that can achieve not only qualitative, but also quantitative assessment of stroke rehabilitation exercises. Specifically, we explored a threshold model that utilizes the outputs of binary classifiers to quantify the correctness of a movements into a performance score. We collected movements of 11 healthy subjects and 15 post-stroke survivors using a Kinect sensor and ground truth scores from primary and secondary therapists. The proposed method achieves the following agreement with the primary therapist: 0.8436, 0.8264, and 0.7976 F1-scores on three task-oriented exercises. Experimental results show that our approach performs equally well or better than multi-class classification, regression, or the evaluation of the secondary therapist. Furthermore, we found a strong correlation (R2 = 0.95) between the sum of computed exercise scores and the Fugl-Meyer Assessment scores, clinically validated motor impairment index of post-stroke survivors. Our results demonstrate a feasibility of automatically assessing stroke rehabilitation exercises with the decent agreement levels and clinical relevance.

In this paper we propose an improved method for transfer learning that takes into account the balance between target and source data. This method builds on the state-of-the-art Multisource Tradaboost, but weighs the importance of each datapoint taking into account the amount of target and source data available. A comparative study is then presented exposing the performance of four transfer learning methods as well as the proposed Weighted Multisource Tradaboost. The experimental results show that the proposed method is able to outperform the base method as the number of target samples increase. These results are promising in the sense that source-target ratio weighing may be a path to improve current methods of transfer learning. However, against the asymptotic conjecture of [6], all transfer learning methods tested in this work get outperformed by a no-transfer SVM for large number on target samples.

Therapists monitor and evaluate stroke patient’s motor abilities with clinical tests to individualize clinical interventions. After a clinical session, a therapist designs task-oriented exercises for a patient and requests self-reporting of patient’s adherence on exercise regimen. However, outpatients, who cannot receive feedback, often show low adherence (Proot et al., 2005) et al, leading to sparse self-reports. It is difficult for therapists to follow patient’s progress. To address this challenge, this paper describes a Kinect-based monitoring system that evaluates performance and provides real-time feedback for four stroke rehabilitation exercises. Our preliminary study showed that this monitoring system can accurately monitor in-home stroke rehabilitation exercises.

In this paper, we present the design and evaluation of our system, which provides an engaging game-based pre-prosthesis training environment for upper limb transradial amputees. We believe that patients who train using such a training tool will demonstrate significantly higher improvement in functional performance tests using a myoelectric prosthesis than when conventional pre-prosthesis training protocols are used. We re-designed two simple games to be playable using three muscle contractions which are appropriate to pre-prosthesis exercises and are detected by an EMG-based arm sleeve. Through user studies conducted with 16 non-amputee subjects, we show that the proposed games are enjoyable, fun to play, sustains motivation to continue playing and also helps in improving muscular control. Using an HMM trained on EMG data collected from the game play sessions of each user, we are able to detect if the subject felt fatigued, while continuously contracting the muscles to play the game, in 93% of the cases and with a latency of only ±15 seconds. We are also able to understand the subject's quality of muscle isolation during the game play.

Action detection and recognition tasks have been the target of much focus in the computer vision community due to their many applications, namely, security, robotics and recommendation systems. Recently, datasets like AVA, provide multi-person, multi-label, spatiotemporal action detection and recognition challenges. Being unable to discern which portions of the input to use for classification is a limitation of two-stream CNN approaches, once the vision task involves several people with several labels. We address this limitation and improve the state-of-the-art performance of two-stream CNNs. In this paper we present four contributions: our fovea attention filtering that highlights targets for classification without discarding background; a generalized binary loss function designed for the AVA dataset; miniAVA, a partition of AVA that maintains temporal continuity and class distribution with only one tenth of the dataset size; and ablation studies on alternative attention filters. Our method, using fovea attention filtering and our generalized binary loss, achieves a relative video mAP improvement of 20% over the two-stream baseline in AVA, and is competitive with the state-of-the-art in the UCF101-24. We also show a relative video mAP improvement of 12.6% when using our generalized binary loss over the standard sum-of-sigmoids.

Presents a listing of the editorial board, board of governors, current staff, committee members, and/or society editors for this issue of the publication.

Automated assessment of fitness exercises has important applications in computer and robot-based exercise coaches to deploy at home, gymnasiums or care centers. In this work, we introduce AHA-3D, a labeled dataset of sequences of 3D skeletal data depicting standard fitness tests on young and elderly subjects, for the purpose of automatic fitness exercises assessment. To the best of our knowledge, AHA-3D is the first publicly available dataset featuring multi-generational, male and female subjects, with frame-level labels, allowing for action segmentation as well as the estimation of metrics like risk of fall, and autonomy to perform daily tasks. We present two baseline methods for recognition and one for segmentation. For recognition, we trained models on the positions of the joints achieving 88.2%± 0.077 accuracy, and on joint positions and velocities, achieving 91%± 0.082 accuracy. Using the Kolmogorov-Smirnov test we determined the model trained on velocities was superior. The segmentation baseline achieved an accuracy of 88.29% in detecting actions at frame level. Our results show promising recognition and detection performance suggesting AHA3D’s potential use in practical applications like exercise performance and correction, elderly fitness level estimation and risk of falling for elders.

When I started at Carnegie Mellon University (CMU) in 1972, I joined Gordon Bell's research group just as he started to explore multiprocessor systems. Later in 1972, Sam Fuller joined the research group and the Computer Module project was born. Cm* (Computer module replicated one or more times as indicated by the Kline star) was an extensible architecture with shared address space across a local bus, extending to a cluster bus, and to intercluster links realizing the first non-uniform memory access (NUMA) multiprocessor [18]. By the end of the 1970s, industry became very interested in multiprocessor systems and it became difficult for university design teams primarily composed of students to keep pace with professional engineering teams with more resources in designing higher speed and ever more complex logic.

Background Ecological momentary assessment (EMA) assesses individuals’ current experiences, behaviors, and moods as they occur in real time and in their natural environment. EMA studies, particularly those of longer duration, are complex and require an infrastructure to support the data flow and monitoring of EMA completion. Objective Our objective is to provide a practical guide to developing and implementing an EMA study, with a focus on the methods and logistics of conducting such a study. Methods The EMPOWER study was a 12-month study that used EMA to examine the triggers of lapses and relapse following intentional weight loss. We report on several studies that informed the implementation of the EMPOWER study: (1) a series of pilot studies, (2) the EMPOWER study’s infrastructure, (3) training of study participants in use of smartphones and the EMA protocol and, (4) strategies used to enhance adherence to completing EMA surveys. Results The study enrolled 151 adults and had 87.4% (132/151) retention rate at 12 months. Our learning experiences in the development of the infrastructure to support EMA assessments for the 12-month study spanned several topic areas. Included were the optimal frequency of EMA prompts to maximize data collection without overburdening participants; the timing and scheduling of EMA prompts; technological lessons to support a longitudinal study, such as proper communication between the Android smartphone, the Web server, and the database server; and use of a phone that provided access to the system’s functionality for EMA data collection to avoid loss of data and minimize the impact of loss of network connectivity. These were especially important in a 1-year study with participants who might travel. It also protected the data collection from any server-side failure. Regular monitoring of participants’ response to EMA prompts was critical, so we built in incentives to enhance completion of EMA surveys. During the first 6 months of the 12-month study interval, adherence to completing EMA surveys was high, with 88.3% (66,978/75,888) completion of random assessments and around 90% (23,411/25,929 and 23,343/26,010) completion of time-contingent assessments, despite the duration of EMA data collection and challenges with implementation. Conclusions This work informed us of the necessary preliminary steps to plan and prepare a longitudinal study using smartphone technology and the critical elements to ensure participant engagement in the potentially burdensome protocol, which spanned 12 months. While this was a technology-supported and -programmed study, it required close oversight to ensure all elements were functioning correctly, particularly once human participants became involved.

Accurate real-time depth camera-based tracking of limbs, fingers and faces would be of great use to the field of Sign Language Recognition (SLR). While aspects of depth-based tracking have been applied to SLR, technological limitations have previously forced trade-offs between the resolution necessary to track finger positions and the field of view necessary to track the signer's body. Only recently, with improvements in cameras and computing power, have algorithms been developed which boast the capability of maintaining accurate finger tracking over an appropriately sized volume of space. In this paper, we employ the publicly available Sphere-Mesh [1] hand tracking algorithm to collect and recognize ASL handshapes. In doing so, we demonstrate recognition rates comparable to other state of the art handshape classifiers using simple naíve Bayesian classifiers that can run in real-time.