Beyond Audio and Video

In this article, we describe the hardware and software challenges involved in realizing claytronics, a form of programmable matter made out of very large numbers—potentially millions—of submillimeter‐sized spherical robots. The goal of the Claytronics Project is to create ensembles of cooperating submillimeter robots, which work together to form dynamic three‐dimensional physical objects. For example, claytronics might be used in telepresense to mimic, with high‐fidelity and in three‐dimensional solid form, the look, feel, and motion of the person at the other end of the telephone call. To achieve this long‐range vision we are investigating hardware mechanisms for constructing submillimeter robots, which can be manufactured en masse using photolithography. We also propose the creation of a new media type, which we call pario. The idea behind pario is to render arbitrary moving, physical three‐dimensional objects that you can see, touch, and even hold in your hands. In parallel with our hardware effort, we are developing novel distributed programming languages and algorithms to control the ensembles, LDP and Meld. Pario may fundamentally change how we communicate with others and interact with the world around us. Our research results to date suggest that there is a viable path to implementing both the hardware and software necessary for claytronics, which is a form of programmable matter that can be used to implement pario. While we have made significant progress, there is still much research ahead in order to turn this vision into reality.