Mobile edge assisted multi-view light field video system: Prototype design and empirical evaluation

Metaverse is recently envisioned as the main driver for immersive multimedia in future networks. Light field video (LFV), considered an intermediate transition scheme towards the metaverse, is conducive to sensing and reconstructing realistic scenes in the digital twins. However, the research on LFV systems still lacks comprehensive investigation, impeding their practical application. This paper addresses major technical challenges involved in LFV delivery through mobile networks, showing a trade-off between equipment cost, calibration effort, and workflow simplicity. It also considers the balance of communications and computations latency when maintaining reconstruction performance. Leveraging edge and cloud infrastructure, a novel Mobile Edge Assisted Multi-view Light-field-video System (MEAMLS) is proposed, integrating LFV collection, volumetric video construction, and an immersive viewing scheme. A custom-made LFV capture array is designed to capture real-world scenes, while employing a learning method-integrated edge server to facilitate adaptive LFV coding. Moreover, a fast sparse reconstruction algorithm is established, leveraging edge-cloud collaboration to minimize computation latency during volumetric video construction. Intelligent service for users is deployed on edge to enable viewport-driven virtual reality viewing. By providing an immersive visual experience, the proposed MEAMLS bridges the physical world and its digital twins. The system is prototyped on a realistic 5G network to empirically validate the performance under static and dynamic circumstances. Experimental results yield fundamental insights for designing mobile edge networks-assisted LFV system from the perspective of transmission resource, adaptability, and deployment.