Combining multiple sensors for event recognition of older people.

ABSTRACTWe herein present a hierarchical model-based framework for event recognition using multiple sensors. Event models combine a priori knowledge of the scene (3D geometric and semantic information, such as contextual zones and equipment) with moving objects (e.g., a Person) detected by a monitoring system. The event models follow a generic ontology based on natural language, which allows domain experts to easily adapt them. The framework novelty relies on combining multiple sensors at decision (event) level, and handling their conflict using a probabilistic approach. The proposed approach for event conflict handling computes the event reliability for each sensor, and then combines them using Dempster-Shafer Theory with an alternative combination rule. The proposed framework is evaluated using multi-sensor recording of instrumental daily living activities (e.g., watching TV, writing a check, preparing tea, organizing week intake of prescribed medication) of participants of a clinical trial for Alzheimer's disease. Two evaluation cases are presented: the combination of events (or activities) from heterogeneous sensors (RGB ambient camera and a wearable inertial sensor) by a deterministic fashion, and the combination of conflicting events recognized by video cameras with partially overlapped field of view (a RGB- and a RGB-D-camera, Kinect®). The results show the framework improves the event recognition rate in both cases.