Developing an Integrated Dashboard to Analyze Multimodal Data for User Experience Evaluation

The evaluation of user experience (UX) is a multifaceted process that encompasses a range of methods such as surveys, emotion recognition, and attention recognition. The effectiveness of UX evaluation depends on the integration of dependable and unbiased techniques derived from diverse user-generated data sources. In this context, we introduce a Multimodal INteractive Dashboard (MIND), a web-based platform tailored to furnish a reliable and user-centric avenue for UX evaluation. MIND incorporates three core functionalities: facial emotion recognition, EEG-based attention recognition, and image generation informed by estimated user’s poses and affective states. By seamlessly engaging users, MIND facilitates a quantitative assessment of their experiences. The proposed MIND offers multimodal data analytics and visualization to enable insights of time spans of the UX with individual users, helping improve the overall user experience.