Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address this problem, we have designed and implemented an open architecture and API for such cameras: the Frankencamera. It consists of a base hardware specification, a software stack based on Linux, and an API for C++. Our architecture permits control and synchronization of the sensor and image processing pipeline at the microsecond time scale, as well as the ability to incorporate and synchronize external hardware like lenses and flashes. This paper specifies our architecture and API, and it describes two reference implementations we have built. Using these implementations we demonstrate six computational photography applications: HDR viewfinding and capture, low-light viewfinding and capture, automated acquisition of extended dynamic range panoramas, foveal imaging, IMU-based hand shake detection, and rephotography. Our goal is to standardize the architecture and distribute Frankencameras to researchers and students, as a step towards creating a community of photographer-programmers who develop algorithms, applications, and hardware for computational cameras.

The FCam API is an open-source camera control library, enabling precise control over a camera's imaging pipeline. Intended for researchers and students in the field of computational photography, it allows easy implementation of novel algorithms and applications. Currently implemented on the Nokia N900 smartphone, and a custom-built "Frankencamera", it has been used in teaching at universities around the world, and is freely available for download for the N900. This paper describes the architecture underlying the API, the design of the API itself, several applications built on top of it, and some examples of its use in education.

From a signal processing perspective, we examine the main factors defining the visual quality of autostereoscopic 3-D displays, which are beginning to reproduce the plenoptic function with increasing accuracy. We propose using intuitive visual tools and ray-tracing simulations to gain insight into the signal processing aspects, and we demonstrate the advantages of analyzing what we call mixed spatial-angular spaces. With this approach we are able to intuitively demonstrate some basic limitations of displays using anisotropic diffusers or lens arrays. Furthermore, we propose new schemes for improved performance.

The ability of a camera to record a high dynamic range image, whether by taking one snapshot or a sequence, is limited by the presence of veiling glare - the tendency of bright objects in the scene to reduce the contrast everywhere within the field of view. Veiling glare is a global illumination effect that arises from multiple scattering of light inside the camera's body and lens optics. By measuring separately the direct and indirect components of the intra-camera light transport, one can increase the maximum dynamic range a particular camera is capable of recording. In this paper, we quantify the presence of veiling glare and related optical artifacts for several types of digital cameras, and we describe two methods for removing them: deconvolution by a measured glare spread function, and a novel direct-indirect separation of the lens transport using a structured occlusion mask. In the second method, we selectively block the light that contributes to veiling glare, thereby attaining significantly higher signal-to-noise ratios than with deconvolution. Finally, we demonstrate our separation method for several combinations of cameras and realistic scenes.

We present a novel technique called symmetric photography to capture real world reflectance fields. The technique models the 8D reflectance field as a transport matrix between the 4D incident light field and the 4D exitant light field. It is a challenging task to acquire this transport matrix due to its large size. Fortunately, the transport matrix is symmetric and often data-sparse. Symmetry enables us to measure the light transport from two sides simultaneously, from the illumination directions and the view directions. Data-sparseness refers to the fact that sub-blocks of the matrix can be well approximated using low-rank representations. We introduce the use of hierarchical tensors as the underlying data structure to capture this data-sparseness, specifically through local rank-1 factorizations of the transport matrix. Besides providing an efficient representation for storage, it enables fast acquisition of the approximated transport matrix and fast rendering of images from the captured matrix. Our prototype acquisition system consists of an array of mirrors and a pair of coaxial projector and camera. We demonstrate the effectiveness of our system with scenes rendered from reflectance fields that were captured by our system. In these renderings we can change the viewpoint as well as relight using arbitrary incident light fields.

Synthetic aperture focusing consists of warping and adding together the images in a 4D light field so that objects lying on a specified surface are aligned and thus in focus, while objects lying of this surface are misaligned and hence blurred. This provides the ability to see through partial occluders such as foliage and crowds, making it a potentially powerful tool for surveillance. If the cameras lie on a plane, it has been previously shown that after an initial homography, one can move the focus through a family of planes that are parallel to the camera plane by merely shifting and adding the images. In this paper, we analyze the warps required for tilted focal planes and arbitrary camera configurations. We characterize the warps using a new rank- 1 constraint that lets us focus on any plane, without having to perform a metric calibration of the cameras. We also show that there are camera configurations and families of tilted focal planes for which the warps can be factorized into an initial homography followed by shifts. This shear-warp factorization permits these tilted focal planes to be synthesized as efficiently as frontoparallel planes. Being able to vary the focus by simply shifting and adding images is relatively simple to implement in hardware and facilitates a real-time implementation. We demonstrate this using an array of 30 videoresolution cameras; initial homographies and shifts are performed on per-camera FPGAs, and additions and a final warp are performed on 3 PCs.

The requirements, features, and implementation of the port interface unit for the Lutonium asynchronous microcontroller are covered in detail. The Lutonium is a new microcontroller that is pin-and software compatible with the Intel 8051 family of microcontrollers, and it is being designed by the Caltech Asynchronous VLSI Group. The port interface unit controls five 8-bit I/O ports, exposing them to the processor as directly accessible registers. The port interface unit also sequences external memory bus access phases using the same I/O ports. The final design allows both backward compatibility with the 8051 series and many advanced power-saving features. It is built around a primary state machine responsible for controlling and sequencing basic operations. The transistor networks for the port unit are complete, and extensive simulations ha ve been done to verify the logical correctness of the design. The layout has not yet been completed, but preliminary performance numbers have been extracted from the transistor-level design.