Ultimately, a display device should be capable of reproducing the visual effects observed in reality. In this paper, we introduce an autostereoscopic display that uses a scalable array of digital light projectors and a projection screen augmented with microlenses to simulate a light field for a given three-dimensional scene. Physical objects emit or reflect light in all directions to create a light field that can be approximated by the light field display. The display can simultaneously provide many viewers from different viewpoints a stereoscopic effect without head tracking or special viewing glasses. This work focuses on two important technical problems related to the light field display: calibration and rendering. We present a solution to automatically calibrate the light field display using a camera and introduce two efficient algorithms to render the special multiview images by exploiting their spatial coherence. The effectiveness of our approach is demonstrated with a four-projector prototype that can display dynamic imagery with full parallax.

Multi-projector systems provide scalability for low-cost, high-resolution display needed for many applications including data visualization, telepresence, and ambient or immersive displays. But geometric calibration of the systems is a great challenge, and for many display configurations, existing camera-based techniques are unsatisfactory. A simple, non-parametric robust technique exists for single-camera calibration, but it doesn't scale to high-resolution displays. Scalability can be gained with a parametric model, but general approaches have tended to compromise robustness or accuracy. Special-purpose models have improved the situation, but at the cost of flexibility. We describe a hybrid calibration framework that combines the robustness of non-parametric calibration with the scalability of parametric calibration. Each parametric calibration technique has a hybrid cousin. The parametric model is fit just as before, but it is not directly used by the rendering algorithm. Instead, the rendering algorithm resembles a non-parametric renderer–it corrects the rendered output using an interpolated lookup table of correspondences between projector framebuffers and a common space. But the space isn't the camera's image; rather it consists of the screen points reconstructed according to the parametric model. By structuring the correspondences and lookup table in a particular way, we are able to ensure that parametric model-fitting error does not hurt the calibration's registration accuracy. We achieve accurate calibration with a general parametric model. Projectors are not modelled, and their radial distortion is corrected implicitly. Multiple cameras are supported. The calibration supports headtracked immersive rendering. A number of related and ancillary research contributions are described in this dissertation, including an analysis of resolution loss in two-pass rendering, together with a technique for preventing the loss by dividing the display surface into clusters based on surface normal. Also techniques are described which improve the reliability of the structure from motion estimation needed for calibration. Other contributions include work in precision feature localization and uncertainty estimation, and a technique for calibrating radial distortion using any number of cameras and without knowledge or assumptions about scene structure. We believe that this work provides key contributions towards robust, hassle-free strong geometric calibration of wide-area, heterogeneous multi-projector environments. Keywords: Projector-Camera Systems, Camera Calibration, Feature Matching, Multi-View Geometry, 3D Reconstruction .

We describe a method for reducing the amount of aliasing or resolution loss in two-pass rendering for distortion and alignment correction of a projector-based display. Resolution loss is caused by the fact that the second rendering pass must resample the result of the first rendering pass, and the two procedures in general have sampling rates that vary differently. We show that for a flat display surface, it is possible to choose a viewing direction for the first-pass render so that its sampling-rate variations cancel with variations of the second-pass sampling rate. This means that the first-pass effectively samples the projector frame-buffer evenly, so an appropriate resolution for the first-pass render will provide uniformly low aliasing over the entire framebuffer. We also show that, for flat display surfaces, this choice of view direction can be combined with an appropriate first-pass intrinsics matrix to eliminate the need for a second pass. The resulting single-pass rendering algorithm is very similar to existing single-pass techniques, but has a few advantages that we discuss. For a non-flat display surface, relative sampling cannot be made perfectly uniform, but the optimal view direction for a best-fit plane provides an approximate solution when the display surface is almost flat. Although the approximation is poor when the display surface differs radically from a plane, we describe a technique for those cases, which automatically subdivides the display surface into partitions that are approximately planar. In common display-surface configurations, great improvements in rendering quality are obtained by using two or three partitions, which causes only modest rendering overhead.

for work related to wide-area video surveillance and human-computer interaction technologies. He is the founder of Mersive Technologies, a company that is commercializing multi-projector display systems and is actively conducting research related to interactive media beyond standard resolutions. Christopher's core research is related to visual information processing, its role in mixed reality and novel display technologies, object recognition and tracking, and intelligent environments. He is the author of over 70 scientific articles, and is the editing author of the book Computer Vision for Interactive and Intelligent Environments (IEEE Press, 2003). He has been the keynote speaker at events ranging from the IEEE Conference on Virtual Reality and Cluster Computing to the Architectural Design conference ACADIA. His research related to multi-projector display systems lead to the formation of Mersive Technologies (www.mersive.com) in 2004 where he currently serves as Chief Technical Officer.

Automatic calibration of multi-projector displays promises a future where very low-cost projectors can be combined into displays whose resolution, size, and fidelity exceeds anything experienced in the past. We broadly refer to these displays as Ultra displays to distinguish them from displays designed for traditional video standards and formats such as High-Definition television. Although ultra displays afford exciting new capabilities, several new challenges must be addressed. In particular, current approaches to rendering, managing and distributing content will have to. We are in the process of exploring these issues as part of an effort to build and deploy a number of ultra displays with vastly different uses. Here we describe the different display prototypes and the challenges that they illuminate for the visualization, computer graphics, and human-computer interaction communities.

Accurate feature detection and localization is fundamentally important to computer vision, and feature locations act as input to many algorithms including camera calibration, structure recovery, and motion estimation. Unfortunately, feature localizers in common use are typically not projectively invariant even in the idealized case of a continuous image. This results in feature location estimates that contain bias which can influence the higher level algorithms that make use of them. While this behavior has been studied in the case of ellipse centroids and then used in a practical calibration algorithm, those results do not trivially generalize to the center-of-mass of a radially symmetric intensity distribution. This paper introduces the generalized result of feature location bias with respect to perspective distortion and applies it to several specific radially symmetric intensity distributions. The impact on calibration is then evaluated. Finally, an initial study is conducted comparing calibration results obtained using center-of-mass to those obtained with an ellipse detector. Results demonstrate that feature localization error, over a range of increasingly large projective distortions, can be stabilized at less than a tenth of a pixel versus errors that can grow to larger than a pixel in the uncorrected case.

Object appearance models are a consequence of illumination, viewing direction, camera intrinsics, and other conditions that are specific to a particular camera. As a result, a model acquired in one view is often inappropriate for use in other viewpoints. In this work we treat this appearance model distortion between two non-overlapping cameras as one in which some unknown color transfer function warps a known appearance model from one view to another. We demonstrate how to recover this function in the case where the distortion function is approximated as general affine and object appearance is represented as a mixture of Gaussians. Appearance models are brought into correspondence by searching for a bijection function that best minimizes an entropic metric for model dissimilarity. These correspondences lead to a solution for the transfer function that brings the parameters of the models into alignment in the UV chromaticity plane. Finally, a set of these transfer functions acquired from a collection of object pairs are generalized to a single camera-pair-specific transfer function via robust fitting. We demonstrate the method in the context of a video surveillance network and show that recognition of subjects in disjoint views can be significantly improved using the new color transfer approach.