Projector-based interactive visual processing

The recent trend of human-computer interaction technologies has revealed the potential of the projector as an powerful interaction tool. More than a pure display tool, a projector has great strength that can change largely the way a traditional user interface works. Although some possibilities have been investigated in previous work, certain applications and approaches deserve further studies. For example, 1) Projection showing 3D information: viewing 3D models is usually achieved by projecting polarized light of different phases for left and right eyes, and the user is required to wear specially designed spectacles. The cost of building such a system is high. 2) Projection on flexible surface: most existing systems display information on flat rigid projection screens, extending it to non-planar flexible surfaces is an interesting and useful research direction; 3) Direct user-info interaction: existing systems using mouse and screen have limited freedom of control and low level of user experience. Direct manipulation of the display object by the hands of a user is more natural; 4) Mobile projector display: portable or embedded projectors are becoming more and more popular, but some fundamental problems, e.g. the keystone correction, are not fully studied. Motivated by these problems, we explore the potential of projectors in interactive information visualization and processing in this thesis. In particular, we make three contributions. First, we propose a computer vision solution for direct 3D object exhibition and manipulation without the user wearing spectacles. In our approach, a new 3D display interface is designed by projecting images on a hand-held foam sphere which can be moved freely by the user. By tracking the motion of the sphere and projecting motion-dependent images onto the sphere, a virtual 3D perception can be created. Using this interface, the user will experience as if he is holding the real object in hands and be able to control the viewing angle freely. Second, we extend the projection on traditional rigid screen to projection on flexible surfaces. A new flexible display method is proposed, which can project information on a hand-held flexible surface (e.g. an ordinary white paper with a checker pattern at the back) that can be twisted freely. While the user twists the projection surface, the system recovers the deformation of the surface and projects well-tailored information onto the surface corresponding to the deformation. As a result, the viewer will see the information as if it was printed on the paper. Two applications, the flexible image projection and curvilinear data slicing are created to demonstrate the usefulness of the method. After the studies on fixed-position projection, we conduct an investigation on mobile projectors, which is becoming especially necessary with the rapid popularity of mobile projectors. We propose a hand-held movable projection method that can freely project keystone-free content onto a general flat surface without any markings or boundaries on the displaying screen. Compared with traditional static projection systems that keep the projector and screen in fixed positions, our projection scheme can give the user greater freedom of display control while producing undistorted images at the same time. To verify the correctness of our methods, we built prototype systems using off-the-shelf devices and conducted extensive experiments, including both simulation and real experiments. The results show that the proposed methods are effective and good performance has been achieved. In particular, the real-time speed and low-cost requirement make it quite appealing in many application areas, such as education, digital games, medical applications etc. Capitalizing on the shrinking size, increasing portability, and decreasing cost of projectors, it is predictable that projector-based interactive processing will become more and more popular in the near future. We believethe research work in this thesis will provide a good foundation for further research and development on computer vision and projector-based applications.