Multi-modal lifelog data fusion for improved human activity recognition: A hybrid approach

The rapid growth of lifelog data, collected through smartphones and wearable devices, has driven the need for better Human Activity Recognition (HAR) solutions. However, lifelog data is complex and challenging to analyze due to its diverse sources of information. In response, we introduce an innovative hybrid data fusion framework for HAR. This framework comprises three key elements: a hybrid fusion mechanism, an attention-based classifier, and an ensemble-based recognition approach. Our hybrid fusion mechanism expertly combines the advantages of late and intermediate fusion, enhancing classification performance and improving the network’s ability to learn connections between different data modalities. Additionally, our solution incorporates an attention-based classifier and an ensemble approach, ensuring robust and consistent performance in real-world scenarios. We evaluated our method across multiple public lifelog datasets, demonstrating that our hybrid fusion approach consistently surpasses existing fusion strategies in HAR, promising significant advancements in activity recognition.