Multiuser Virtual Experiences Powered by Holoportation Technologies and Multimodal Human-Computer Interaction (HCI)

This paper elaborates on a set of research objectives to optimize multi-user Social Virtual Reality (VR) experiences. A first objective consists of designing and developing a novel and lightweight Volumetric Video (VV) pipeline to provide 3D realistic user representations (i.e., holograms), increasing the visual resolution and real-time performance compared to state-of-the-art solutions. A second objective consists of designing and implementing novel modular communication architectures, integrating virtualized in-cloud media processing functions with the goal of enhancing the scalability and interoperability in Social VR. A third objective consists of designing and implementing a set of multimodal Human Computer Interaction (HCI) features to be integrated in Social VR experiences to enable rich group-based interaction and effective collaboration. All such envisioned contributions will be thoroughly (objectively and subjectively) evaluated in relevant use cases, like culture and tourism, to validate their benefits and shed some light on associated hypotheses.