This paper explores the optimization of light field deconvolution, a key process in image processing that reconstructs a 3D object space or a 2D refocus plane from a light field. Despite the critical role of deconvolution in light field technology, existing methods are often slow, computationally intensive, and unsuitable for real-time processing. Existing algorithms, such as the Richardson-Lucy approach, while groundbreaking, still suffer performance limitations due to their iterative nature and high computational costs. Central to our approach is the strategic selection of influential pixels within the point-spread-function, reducing redundant computations by focusing only on pixels contributing to a significant portion of the point-spread-function’s total intensity. In addition, we explore the potential to directly invert the image formation model, bypass iterative computations, and further accelerate the deconvolution process. Our findings reveal notable improvements in computational efficiency, with some of our methods achieving real-time performance. The reconstruction quality, measured using metrics such as the mean squared error, remained comparable to existing approaches, indicating a favorable balance between speed and reconstruction quality.

Conventional microscopes have a high spatial resolution and a low depth-of-field. Light field microscopes have a high depth-of-field but low spatial resolution. A new hybrid approach uses information from both systems to reconstruct a high-resolution light field [Appl. Opt. 58, A142 (2019)APOPAI0003-693510.1364/AO.58.00A142]. The resolution of the resulting light field is said to be limited only by diffraction and the size of the pixels. In this paper, we evaluate this method. Using simulation data we compare the output of the hybrid reconstruction algorithm with its simulated ground truth. Our analyses reveal that the observed improvement in the light field quality is not a consequence of data fusion or incorporation of information from a conventional camera, but rather the results of an intermediate interpolation step within the light field itself. This suggests that the required information is already inherent to the light field. By employing the Richardson-Lucy Light Field Deconvolution algorithm, we demonstrate that existing algorithms have already utilized this information.

Plenoptic cameras capture the spatial and angular information of a scene. The use of plenoptic cameras in areas such as research, microscopy, industry and consumer markets has steadily increased over the past two decades. When designing a plenoptic camera a decision must always be made between spatial and angular resolution. Many factors such as the size and number of microlenses in a microlens array or the relative position of the microlens array and sensor to the main lens play a role. Here we examine the two most common designs of plenoptic cameras. The plenoptic cameras 1.0, also called the unfocused plenoptic camera, and the plenoptic cameras 2.0, the focused plenoptic camera. We derive the mathematical equations that describe the connection between spatial and angular resolution. Supported by experimental results we show the relationship between the equations and a real object with different object distances taken from the plenoptic camera 1.0 and 2.0. These analyzes make it easier for researchers and engineers to choose the right camera design for a particular application. The user only has to determine beforehand which depth-of-field or which spatial resolution is needed.

Automated re-identification (re-id) of vehicles is the foundation of many traffic analysis applications across camera networks, e.g. vehicle tracking, counting, or traffic density and flow estimation. The re-id task is made difficult by variations in lighting, viewpoint, image quality and similar vehicle models and colors that can occur across the network. These influences can cause a high visual appearance variation for the same vehicle while different vehicles may appear near identical under similar conditions. However, with a growing number of available datasets and well crafted deep learning models, much progress has been made. In this work we summarize the results of our participation in the NVIDIA AI City Challenge 2020 for vehicle reid. We address the re-id task by relying on well-proven design choices from the closely related person re-id literature. In addition to this, we explicitly address viewpoint and occlusions variation. The former is addressed by incorporating vehicle viewpoint classification results into our matching distance. The required viewpoint classifier is trained predominantly on simulated data and we show that it can be applied to real-world imagery with minimal domain adaptation. We address occlusion by relying on temporal attention scores which emphasize video frames that contain minimal occlusion. Finally, we further boost re-id accuracy by applying video-based re-ranking and an ensemble of complementary models. Our models, code, and simulated data is available at https://github.com/corner100/2020-aicitychallenge-IOSB-VeRi.

Calibration of wide field-of-view cameras is a fundamental step for numerous visual media production applications, such as 3D reconstruction, image undistortion, augmented reality and camera motion estimation. However, existing calibration methods require multiple images of a calibration pattern (typically a checkerboard), assume the presence of lines, require manual interaction and/or need an image sequence. In contrast, we present a novel fully automatic deep learning-based approach that overcomes all these limitations and works with a single image of general scenes. Our approach builds upon the recent developments in deep Convolutional Neural Networks (CNN): our network automatically estimates the intrinsic parameters of the camera (focal length and distortion parameter) from a single input image. In order to train the CNN, we leverage the great amount of omnidirectional images available on the Internet to automatically generate a large-scale dataset composed of millions of wide field-of-view images with ground truth intrinsic parameters. Experiments successfully demonstrated the quality of our results, both quantitatively and qualitatively.