Edge-Based Cross-Modal Communications for Remote Healthcare

Medical robots with audio-video-haptic streams, as indispensable devices for remote healthcare, are playing ever-increasing roles in mitigating the spread of infectious diseases. However, existing medical robots are far from precise and efficient because of the following two technical challenges, including i) how to ensure the haptic fidelity for precise manipulation, and ii) how to alleviate the impact of haptic streams on the quality of visual navigation for efficient operation. To this end, this work explores the benefits of edge-based cross-modal communications (CMCs), which take full advantage of potential correlations among different modalities’ streams, to realize high reliability and throughput. Specifically, to compensate for the reliability loss caused by wireless transmission, a semantic-aided cross-modal reconstruction framework is firstly designed at edge nodes for high haptic fidelity. Then, a user experience-driven stream scheduling strategy is developed to enhance the visual quality by fully leveraging edge computing and network slicing. In particular, different from traditionally interrupting audio/video stream transmission to prioritize haptic streams, we jointly schedule resources to different modalities’ streams via estimating haptic arrival time. Finally, as a classical case study, we independently construct a remote throat swab sampling platform based on CMCs to evaluate practical performance, and numerical results indicate the significant improvements in terms of various metrics.