Phygital Field: An Integrated Field with Physical Robots and Digital Images Using Projection-Based Localization and Control Method

Collaboration between computer graphics and multiple robots has attracted increasing attention in several fields. To enhance the seamless connection between them, the system should be able to accurately determine the position and state of the robots and to control them easily and instantly. However, realizing a responsive control system for a large number of mobile robots without complicated settings while avoiding the system load problem is not trivial. We propose a novel system, called “Phygital Field,” for the localization and control of multiple mobile robots. Utilizing pixel-level visible light communication technology, our system can project two types of information in the same location: visible images for humans and data patterns for mobile robots. The system uses coded light superimposed onto a visual image and projected onto the robots. The robots localize their position by receiving and decoding the projected light and can follow a target using the coded velocity vector field. Localization and control information can be independently conveyed in each pixel, and we can change this information over time. The system only requires a projector to control the robot swarm; thus, it can be used on any projection surface. We experimentally assess the localization accuracy of our system for both stationary and moving robots. To further illustrate the utility of our proposed system, we demonstrate the control of multiple mobile robots in spatially and temporally varying vector fields. We also propose prototype applications that can provide users with novel content from collaboration between computer graphics and robot swarm.