A real-time performance system for virtual theater.

The idea of combining virtual reality technology and theatrical tradition to create virtual plays has captured artists' imaginations for some time. Using conventional technology, the use of virtual characters in a theatrical performance often integrates the predefined animations of virtual actors into the theater scene, resulting in a performance that can feel stilted and unresponsive due to its pre-programmed nature. This paper proposes a new system that allows actors to animate virtual characters in real time, resulting in a more flexible and interactive theatrical performance experience. Actors are sequestered at a remote site, invisible to the audience, and are digitized by a motion capture system. Using camera feeds to provide the remote actors with information about the behavior of the live actors and audience in the theater, the remote actors can adapt their virtual counterparts' behavior to react to live events in real-time, giving the illusion to the audience that the virtual characters are responsive to their actions. The system integrates display peripherals, networked cameras, real-time motion capture system, gesture recognition, and a virtual environment development suite (Virtools) to create a true virtual theatrical performance environment. In the paper, we will present the various concepts developed so far and an example of a virtual theatrical performance called Trickster at the Intersection that was presented during Smart Graphics 2010 at Canada's Banff Centre.