Video surveillance is an alternative approach to staff or self- reporting that has the potential to detect and monitor aggressive behaviors more accurately. In this paper, we propose an automatic algorithm capable of recognizing aggressive behaviors from video records using local binary motion descriptors. The proposed algorithm will increase the accuracy for retrieving aggressive behaviors from video records, and thereby facilitate scientific inquiry into this low frequency but high impact phenomenon that eludes other measurement approaches.

Data for diagnosis and clinical studies are now typically gathered by hand. While more detailed, exhaustive behavioral assessments scales have been developed, they have the drawback of being too time consuming and manual assessment can be subjective. Besides, clinical knowledge is required for accurate manual assessment, for which extensive training is needed. Therefore our great research challenge is to leverage machine learning techniques to better understand patients health status automatically based on continuous computer observations. In this paper, we study the problem of health status prediction for geriatric patients using observational data. In the first part of this paper, we propose a distance metric learning algorithm to learn a Mahalanobis distance which is more precise for similarity measures. In the second part, we propose a robust classifier based on ℓ 2,1 -norm regression to predict the geriatric patients' health status. We test the algorithm on a dataset collected from a nursing home. Experiment shows that our algorithm achieves encouraging performance.

As our society is increasingly aging, it is urgent to develop computer aided techniques to improve the quality-of-care (QoC) and quality-of-life (QoL) of geriatric patients. In this paper, we focus on automatic human activities analysis in video surveillance recorded in complicated environments at a nursing home. This will enable the automatic exploration of the statistical patterns between patients’ daily activities and their clinical diagnosis. We also discuss potential future research directions in this area. Experiment demonstrate the proposed approach is effective for human activity analysis.

This chapter contains sections titled: Roles Clinical Issues Regulations References

Cognitive impairment and frailty associated with dementia renders residents of long-term care (LTC) facilities particularly vulnerable to physical and emotional harm. Resident-to-resident violence affects not only the target of the aggression, but also the aggressor, as well as the formal and informal caregivers who must intervene. To date, little research has been conducted on resident-to-resident violence despite preliminary but emerging evidence that it is a common (and likely growing) problem in LTC settings. Exploration of this phenomenon presents multiple pragmatic and ethical challenges. This article presents a rationale for implementing newer technological methods to collect data in investigations of resident-to-resident violence associated with dementia. The advantages and disadvantages of electronic surveillance in LTC research and the ethical principles involved are discussed, and an argument is developed for using electronic surveillance in both the shared, as well as private, spaces of the facility.

The number of older Americans afflicted by Alzheimer disease and related dementias will triple to 13 million persons by 2050, thus greatly increasing healthcare needs. An approach to this emerging crisis is the development and deployment of intelligent assistive technologies that compensate for the specific physical and cognitive deficits of older adults with dementia, and thereby also reduce caregiver burden. The authors conducted an extensive search of the computer science, engineering, and medical databases to review intelligent cognitive devices, physiologic and environmental sensors, and advanced integrated sensor networks that may find future applications in dementia care. Review of the extant literature reveals an overwhelming focus on the physical disability of younger persons with typically nonprogressive anoxic and traumatic brain injuries, with few clinical studies specifically involving persons with dementia. A discussion of the specific capabilities, strengths, and limitations of each technology is followed by an overview of research methodological challenges that must be addressed to achieve measurable progress to meet the healthcare needs of an aging America.

ASSISTIVE TECHNOLOGIES ARE RELATIVELY novel tools for research and daily care in long-term care (LTC) facilities that are faced with the burgeoning of the older adult population and dwindling staffing resources. The degree to which stakeholders in LTC facilities are receptive to the use of these technologies is poorly understood. Eighteen semi-structured focus groups and one interview were conducted with relevant groups of stakeholders at seven LTC facilities in southwestern Pennsylvania. Common themes identified across all focus groups centered on concerns for privacy, autonomy, cost, and safety associated with implementation of novel technologies. The relative importance of each theme varied by stakeholder group as well as the perceived severity of cognitive and/or physical disability. Our findings suggest that ethical issues are critical to acceptance of novel technologies by their end users, and that stakeholder groups are interdependent and require shared communication about the acceptance of these emerging technologies.

Objective To examine the total and domain-specific prevalence of verbally and physically abusive, socially inappropriate, and care-resistive behaviors according to the Minimum Data Set (MDS) compared with research instruments in nursing home residents with severe dementia. Design, Setting, and Methods As part of a longitudinal observational study, MDS behavioral symptoms data were compared with corresponding items from the Ryden Aggression Scale and the Cohen-Mansfield Agitation Inventory for 15 nursing home residents with severe dementia. McNemar's test was used to compare the difference in the proportion of subjects who experienced any symptoms, as well as specific symptoms in several domains, according to the MDS and the research instruments. Additionally, temporal fluctuations in behavioral symptoms were descriptively and graphically summarized. Results The MDS significantly underestimated both the total proportion of subjects experiencing any behavioral symptoms (P = .016), as well as the proportion of subjects experiencing verbally abusive symptoms (P < .002), physically abusive symptoms (P = .008), or socially inappropriate behaviors (P = .016) compared with corresponding items from the research instruments. Moreover, these behaviors exhibited considerable temporal instability, suggesting that the systematic daily collection of measures of behavioral disturbances is imperative during the week in which the MDS assessment is to be completed. Discussion Albeit from a small study sample, our findings call into question the validity of the MDS behavioral symptom items as they are currently recorded, and suggest that a simple intervention of twice daily completion of a behavioral symptoms checklist containing the MDS items during the week of the assessment may significantly improve the accuracy of the recorded data.

Video surveillance is an alternative approach to staff or self-reporting that has the potential to detect and monitor aggressive behaviors more accurately. In this paper, we propose an automatic algorithm capable of recognizing aggressive behaviors from video records using local binary motion descriptors. The proposed algorithm may increase the accuracy for retrieving aggressive behaviors from video records, and thereby facilitates scientific inquiry into this low frequency but high impact phenomenon that eludes other measurement approaches.

Automated detection of aggressive behaviors captured in continuously recorded nursing home video can increase the accuracy of those reported by subjects and caregivers. We implement a detection algorithm based on the extraction of features as local binary motion descriptors, and apply a novel clustering algorithm to merge similar LBMD's into a video codebook, and build recognizers to classify aggressive from non-aggressive behaviors. This facilitates clinical investigation into this difficult to measure low frequency but high impact behavior.

OBJECTIVES: To more fully characterize the spectrum of resident‐to‐resident aggression (RRA). DESIGN: A focus group study of nursing home staff members and residents who could reliably self‐report. SETTING: A large, urban, long‐term care facility. PARTICIPANTS: Seven residents and 96 staff members from multiple clinical and nonclinical occupational groups. MEASUREMENTS: Sixteen focus groups were conducted. Content was analyzed using nVivo 7 software for qualitative data. RESULTS: Thirty‐five different types of physical, verbal, and sexual RRA were described, with screaming or yelling being the most common. Calling out and making noise were the most frequent of 29 antecedents identified as instigating episodes of RRA. RRA was most frequent in dining and residents' rooms, and in the afternoon, although it occurred regularly throughout the facility at all times. Although no proven strategies exist to manage RRA, staff described 25 self‐initiated techniques to address the problem. CONCLUSION: RRA is a ubiquitous phenomenon in nursing home settings, with important consequences for affected individuals and facilities. Further epidemiological research is necessary to more fully describe the phenomenon and identify risk factors and preventative strategies.

This paper presents the application of computer vision and machine learning technologies to a clinical task of paramount importance, improving safety of older persons. We propose an intelligent monitoring system equipped with a camera network and an automatic elopement detection algorithm to reduce the risks of un-witnessed elopements from a dementia unit in order to avoid their potential catastrophic consequences. The camera network employs 23 cameras to record daily activities in our test bed, which includes 15 residents, 4 registered and licensed practical nurses and a number of certified nursing assistants. An elopement detector is then built by using computer vision algorithms and a machine learning algorithm to automatically detect elopements and alert caregivers. The experiments demonstrate that the proposed system leverages the advantages of monitoring from multiple cameras and is able to detect elopements with almost 100% accuracy.

In the absence of objective, reliable assessment and outcomes measurement methodologies in a nursing home, effectiveness of behavioral and pharmacological interventions cannot be determined. Pervasive technology holds the promise of developing objective, real-time, continuous assessment and outcomes measurement methodologies that were previously unfeasible. Such technologies can contribute greatly to a deeper understanding of the activity and behavior patterns of individual residents, and the physical, environmental and psychosocial correlates of these patterns. Bharucha, A., Allin, S. and Stevens, S., “CareMedia: Towards Automated Behavior Analysis in the Nursing Home Setting,” in The International Psychogeriatric Association Eleventh International Conference, Aug. 17-22, 2003. Several hours of surveillance-type video were captured in a nursing home. The task of data reduction and extraction of highlevel activity information was approached through both automated and manual techniques. For the manual encoding, 4 undergraduate students were trained by a geriatric psychiatrist to code the data frame-by-frame. A computer interface allowed coders to annotate behaviors of interest, as well as physical pose and ambulatory status. Behaviors of interest were identified with the CohenMansfield Agitation Inventory and grouped into 4 sub-categories: CareMedia Carnegie Mellon University 8 CareMedia: Automated Video and Sensor Analysis for Geriatric Care March 2003 Annual Progress Report physically aggressive, physically non-aggressive, verbally aggressive, and verbally non-aggressive. These manual encodings are currently forming the development of automated techniques at Carnegie Mellon University to extract information relevant to the detection of anomalous and disruptive physical activities. This includes automated tracking and extraction of navigational patterns. Gao, J., Hauptmann, A.G., Barucha, A. and Wactlar, H.D., “Dining Activity Analysis Using Hidden Markov Models,” accepted to The 17th International Conference on Pattern Recognition (ICPR’04), Cambridge, United Kingdom, Aug. 23-26, 2004. Abstract: We describe an algorithm for dining activity analysis in a nursing home. Based on several features, including motion vectors and distance between moving regions in the subspace of an individual person, a hidden Markov model is proposed to characterize different stages in dining activities with certain temporal order. Using HMM model, we are able to identify the start (and ending) of individual dining events with high accuracy and low false positive rate. This approach could be successful in assisting caregivers in assessments of resident's activity levels over time. We describe an algorithm for dining activity analysis in a nursing home. Based on several features, including motion vectors and distance between moving regions in the subspace of an individual person, a hidden Markov model is proposed to characterize different stages in dining activities with certain temporal order. Using HMM model, we are able to identify the start (and ending) of individual dining events with high accuracy and low false positive rate. This approach could be successful in assisting caregivers in assessments of resident's activity levels over time. Gao, J., Hauptmann, A.G. and Wactlar, H.D., “Combining Motion Segmentation with Tracking for Activity Analysis,” submitted to The Sixth International Conference on Automatic Face and Gesture Recognition (FG’04), Seoul, Korea, May 17-19, 2004. Abstract: We explore a novel motion feature as the appropriate basis for classifying or describing a number of fine motor human activities. Our approach not only estimates motion directions and magnitudes in different image regions, but also provides accurate segmentation of moving regions. Through a combination of motion segmentation and region tracking techniques, while filtering for temporal consistency, we achieve a balance between accuracy and reliability of motion feature extraction. To identify specific activities, we characterize the dominant directions of relative motions. Experimental results show that this approach to motion feature analysis could be successful in assisting caregivers at a nursing home in assessments of patient's activity levels over time. We explore a novel motion feature as the appropriate basis for classifying or describing a number of fine motor human activities. Our approach not only estimates motion directions and magnitudes in different image regions, but also provides accurate segmentation of moving regions. Through a combination of motion segmentation and region tracking techniques, while filtering for temporal consistency, we achieve a balance between accuracy and reliability of motion feature extraction. To identify specific activities, we characterize the dominant directions of relative motions. Experimental results show that this approach to motion feature analysis could be successful in assisting caregivers at a nursing home in assessments of patient's activity levels over time. Hauptmann, A.G., Gao, J., Yan, R., Qi, Y., Yang, J., and Wactlar, H.D., “Aiding Geriatric Patients and Caregivers through Automated Analysis of Nursing Home Observations,” to be published in IEEE Pervasive Computing, April-June special issue: Pervasive Computing for Successful Aging. Abstract: Through pervasive activity monitoring in a skilled nursing facility, a continuous audio and video record is captured. Through pervasive activity monitoring in a skilled nursing facility, a continuous audio and video record is captured. CareMedia Carnegie Mellon University 9 CareMedia: Automated Video and Sensor Analysis for Geriatric Care March 2003 Annual Progress Report Our CareMedia Project research analyzes this video information by automatically tracking people, assisting in efficiently labeling individuals, and characterizing selected activities and actions. Special emphasis is given to detecting eating activity in the dining hall and to personal hygiene. Through this work, the video record is transformed into an information asset that can provide geriatric care specialists with greater insights and evaluation of behavioral problems for the elderly. Evaluations of the effectiveness of analyzing such a large video record illustrate the feasibility of our approach. Hauptmann, A.G., Jin, R. and Wactlar, H.D., “Data Analysis for a Multimedia Library, in Text and Speech-Triggered Information Access,” Renals, S and Grefenstette, G. (eds)., Springer, Berlin, pp. 6-37, 2003. Abstract: This book section describes the indexing, search and retrieval of various combinations of audio, video, text and image media and the automated content processing that enables it. The intent is to provide a framework for data analysis in multimedia digital libraries. The introduction briefly distinguishes the digital from traditional libraries and touches on the specific issues important to searching the content of multimedia libraries. The second section introduces the Informedia Digital Video Library as an example of a multimedia library, including a quick tour of the functionality. The next section discusses the processing of audio and image information, as it relates to a multimedia library. Section four illustrates the interplay between audio and video information using a video information retrieval experiment as an example. Section five discusses the exporting and sharing of metadata in a digital library using MPEG-7. Finally, section 6 provides one vision of a future digital library, where all personal memory can be recorded and accessed. This book section describes the indexing, search and retrieval of various combinations of audio, video, text and image media and the automated content processing that enables it. The intent is to provide a framework for data analysis in multimedia digital libraries. The introduction briefly distinguishes the digital from traditional libraries and touches on the specific issues important to searching the content of multimedia libraries. The second section introduces the Informedia Digital Video Library as an example of a multimedia library, including a quick tour of the functionality. The next section discusses the processing of audio and image information, as it relates to a multimedia library. Section four illustrates the interplay between audio and video information using a video information retrieval experiment as an example. Section five discusses the exporting and sharing of metadata in a digital library using MPEG-7. Finally, section 6 provides one vision of a future digital library, where all personal memory can be recorded and accessed. Jin, R., Hauptmann, A., Carbonell, J., Si, L., Liu, Y., “A New Boosting Algorithm Using Input Dependent Regularizer,” 20th International Conference on Machine Learning (ICML'03), Washington, DC, August 21-24, 2003. Abstract: AdaBoost has proved to be an effective method to improve the performance of base classifiers both theoretically and empirically. However, previous studies have shown that AdaBoost might suffer from the overfitting problem, especially for noisy data. In addition, most current work on boosting assumes that the combination weights are fixed constants and therefore does not take particular input patterns into consideration. In this paper, we present a new boosting algorithm, “WeightBoost”, which tries to solve these two problems by introducing an input-dependent regularization factor to the combination weight. Similarly to AdaBoost, we derive a learning procedure for WeightBoost, which AdaBoost has proved to be an effective method to improve the performance of base classifiers both theoretically and empirically. However, previous studies have shown that AdaBoost might suffer from the overfitting problem, especially for noisy data. In addition, most current work on boosting assumes that the combination weights are fixed constants and therefore does not take particular input p

Innovative technologies are rapidly emerging that offer caregivers the support and means to assist older adults with cognitive impairment to continue living "at home." Technology research and development efforts applied to older adults with dementia invoke special grant review and institutional review board concerns, to ensure not only safe but also ethically appropriate interventions. Evidence is emerging, however, that tensions are growing between innovators and reviewers. Reviewers with antitechnology biases are in a position to stifle needed innovation. Technology developers who fail to understand the clinical and caregiving aspects of dementia may design applications that are not in alignment with users' capabilities. To bridge this divide, we offer an analysis of the ethical issues surrounding home monitoring, a model framework, and ethical guidelines for technology research and development for persons with Alzheimer's disease and their caregivers.

Four cases reveal the main themes: "taking care" included mutual protection between patients and family members; "midwifing the death" without professional support left families unprepared for adverse events; "tying up loose ends" included dealing with family members' fear of legal consequences; and "moving ahead" involved a greater risk of complicated grief when families encountered complications during the dying process. These results highlight the positive and negative consequences of family members' participation in a hastened

CareMedia is a collaborative effort that to date has captured more than 13,000 hours of video and audio recordings of life in the shared spaces of a nursing home dementia unit, by using 23 ceiling-mounted cameras, 24 hours a day for 25 days, ensuring an un-occluded view of every point in the recorded space. Computer machine learning techniques are being applied to the resulting 25 Terabytes of data, automatically processing the record for efficient use by analytical observers (e.g., social and behavioral scientists, geriatricians, and healthcare policy makers) to monitor and understand residents' well-being, and enhance their quality of life. This truly interdisciplinary effort bridges the psychological, social and behavioral sciences, and clinical medicine with multiple engineering and computer science disciplines to establish a clinical evidence base to guide rational therapeutics, an elusive goal ardently articulated by the Institute of Medicine. This paper discusses early foundation work being conducted with the data.