Stereo imaging is the most common passive method for producing reliable depth maps. Calibration is a crucial step for every stereo-based system, and despite all the advancements in the field, most calibrations are still done by the same tedious method using a checkerboard target. Monocular-based depth estimation methods do not require extrinsic calibration but generally achieve inferior depth accuracy. In this paper, we present a novel online self-calibration method, which makes use of both stereo and monocular depth maps to find the transformation required for extrinsic calibration by enforcing consistency between both maps. The proposed method works in a closed-loop and exploits the pre-trained networks’ global context, and thus avoids feature matching and outliers issues. In addition to presenting our method using an image-based monocular depth estimation method, which can be implemented in most systems without additional changes, we also show that adding a phase-coded aperture mask leads to even better and faster convergence. We demonstrate our method on road scenes from the KITTI vision benchmark and real-world scenes using our prototype camera. Our code is publicly available at https://github.com/YotYot/CalibrationNet.

Motion blur is a known issue in photography, as it limits the exposure time while capturing moving objects. Extensive research has been carried to compensate for it. In this work, a computational imaging approach for motion deblurring is proposed and demonstrated. Using dynamic phase-coding in the lens aperture during the image acquisition, the trajectory of the motion is encoded in an intermediate optical image. This encoding embeds both the motion direction and extent by coloring the spatial blur of each object. The color cues serve as prior information for a blind deblurring process, implemented using a convolutional neural network (CNN) trained to utilize such coding for image restoration. We demonstrate the advantage of the proposed approach over blind-deblurring with no coding and other solutions that use coded acquisition, both in simulation and real-world experiments.

Motion deblurring solution based on spatio-temporal coding is proposed. Using aperture coding and focus variations during exposure, a joint spatio-temporal coding is achieved, which is in-turn utilized for motion deblurring in the post processing step.

Passive depth estimation is among the most long-studied fields in computer vision. The most common methods for passive depth estimation are either a stereo or a monocular system. Using the former requires an accurate calibration process, and has a limited effective range. The latter, which does not require extrinsic calibration but generally achieves inferior depth accuracy, can be tuned to achieve better results in part of the depth range. In this work, we suggest combining the two frameworks. We propose a two-camera system, in which the cameras are used jointly to extract a stereo depth and individually to provide a monocular depth from each camera. The combination of these depth maps leads to more accurate depth estimation. Moreover, enforcing consistency between the extracted maps leads to a novel online self-calibration strategy. We present a prototype camera that demonstrates the benefits of the proposed combination, for both self-calibration and depth reconstruction in real-world scenes.

Depth estimation from a single image is a well-known challenge in computer vision. With the advent of deep learning, several approaches for monocular depth estimation have been proposed, all of which have inherent limitations due to the scarce depth cues that exist in a single image. Moreover, these methods are very demanding computationally, which makes them inadequate for systems with limited processing power. In this paper, a phase-coded aperture camera for depth estimation is proposed. The camera is equipped with an optical phase mask that provides unambiguous depth-related color characteristics for the captured image. These are used for estimating the scene depth map using a fully convolutional neural network. The phase-coded aperture structure is learned jointly with the network weights using backpropagation. The strong depth cues (encoded in the image by the phase mask, designed together with the network weights) allow a much simpler neural network architecture for faster and more accurate depth estimation. Performance achieved on simulated images as well as on a real optical setup is superior to the state-of-the-art monocular depth estimation methods (both with respect to the depth accuracy and required processing power), and is competitive with more complex and expensive depth estimation methods such as light-field cameras.

Single image depth estimation is achieved using computational imaging and Deep Learning (DL). Imaging with phase-mask is also modeled as a DL-layer, and the mask and DL parameters are jointly designed using labeled data.

Modern consumer electronics market dictates the need for small-scale and high-performance cameras. Such designs involve trade-offs between various system parameters. In such trade-offs, Depth Of Field (DOF) is a significant issue very often. We propose a computational imaging-based technique to overcome DOF limitations. Our approach is based on the synergy between a simple phase aperture coding element and a convolutional neural network (CNN). The phase element, designed for DOF extension using color diversity in the imaging system response, causes chromatic variations by creating a different defocus blur for each color channel of the image. The phase-mask is designed such that the CNN model is able to restore from the coded image an all-in-focus image easily. This is achieved by using a joint end-to-end training of both the phase element and the CNN parameters using backpropagation. The proposed approach provides superior performance to other methods in simulations as well as in real-world scenes.

Lenses used in many infrared (IR) imaging systems are temperature sensitive. One of the most popular IR optical materials for lens fabrication is germanium; nevertheless, it exhibits a strong temperature dependent refractive index, causing significant thermal focal shift which in turn results in image blur. An all-optical solution for IR lens athermalization with no moving parts based on a thermally dependent binary phase mask is hereby proposed and analyzed. It allows high quality imaging to be obtained for a wide range of temperature variations, with minimal performance degradation at nominal temperature conditions.

Design of Infra-red (IR) imaging systems requires solutions for overcoming thermal variations of IR lenses, in particular those fabricated out of Germanium. The known thermal dependence of the index of refraction of Germanium results in significant focal shift, which produces image blur. Known solutions to overcome the thermal fluctuations are reviewed. A solution based on an alloptical phase mask with no moving parts will be shown to provide improved imagery for broad band IR scenes in the presence of wide temperature variations. Phase mask design considerations and trade-offs are analyzed.

A method for extended depth of field imaging based on image acquisition through a thin binary phase plate followed by fast automatic computational post-processing is presented. By placing a wavelength dependent optical mask inside the pupil of a conventional camera lens, one acquires a unique response for each of the three main color channels, which adds valuable information that allows blind reconstruction of blurred images without the need of an iterative search process for estimating the blurring kernel. The presented simulation as well as capture of a real life scene show how acquiring a one-shot image focused at a single plane, enable generating a de-blurred scene over an extended range in space. ©2015 Optical Society of America OCIS codes: (110.1758) Computational imaging; (110.1455) Blind deconvolution; (110.3010) Image reconstruction techniques; (100.3020) Image reconstruction-restoration; (100.3190) Inverse problems References and links 1. B. Milgrom, N. Konforti, M. A. Golub, and E. Marom, “Pupil coding masks for imaging polychromatic scenes with high resolution and extended depth of field,” Opt Express, 18 (15), 15569–15584, (2010). 2. B. Milgrom, N. Konforti, M. A. Golub, and E. Marom, “Novel approach for extending the depth of field of Barcode decoders by using RGB channels of information,” Opt Express, 18 (16), 17027–17039, (2010). 3. J. L. Starck, E. Pantin, and F. Murtagh, “Deconvolution in Astronomy: A Review,” Publ. Astron. Soc. Pacific, The University of Chicago Press, 114 (800), 1051–1069, (2002). 4. M. Elad, Sparse and redundant representations : from theory to applications in signal and image processing. (Springer, 2010). 5. M. Elad and M. Aharon, “Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries,” IEEE Trans. Image Process., 15 (12), 3736–3745, (2006). 6. M. J. Fadili, J. L. Starck, and F. Murtagh, “Inpainting and zooming using sparse representations,” Comput. J., 52 (1), 64–79, (2009). 7. F. Couzinie-Devy, J. Mairal, F. Bach, and J. Ponce, “Dictionary learning for deblurring and digital zoom,” arXiv Prepr. arXiv1110.0957, (2011). 8. M. S. C. Almeida and L. B. Almeida, “Blind and semi-blind deblurring of natural images.,” IEEE Trans. Image Process., 19 (1), 36–52, (2010). 9. Q. Shan, J. Jia, and A. Agarwala, “High-quality motion deblurring from a single image,” in ACM Transactions on Graphics (TOG), ACM, 27 (3), pp. 73. 10. Z. Hu, J. Bin Huang, and M. H. Yang, “Single image deblurring with adaptive dictionary learning,” in Image Processing (ICIP), 2010 17th IEEE International Conference on, IEEE, pp. 1169–1172. 11. D. Krishnan, T. Tay, and R. Fergus, “Blind deconvolution using a normalized sparsity measure,” in Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, IEEE, pp. 233–240. 12. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a handheld plenoptic camera,” Comput. Sci. Tech. Rep. CSTR, 2 (11), (2005). 13. A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. Graph., 26 (3), 70, (2007). 14. F. Guichard, H.-P. Nguyen, R. Tessières, M. Pyanet, I. Tarchouna, and F. Cao, “Extended depth-of-field using sharpness transport across color channels,” in IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics (2009), pp. 72500N–72500N–12. 15. J. W. Goodman, Introduction to Fourier optics, 2nd ed. (McGraw-Hill, 1996).

A method for extended depth of field imaging based on image acquisition through a thin binary phase plate followed by fast automatic computational post-processing is presented. By placing a wavelength dependent optical mask inside the pupil of a conventional camera lens, one acquires a unique response for each of the three main color channels, which adds valuable information that allows blind reconstruction of blurred images without the need of an iterative search process for estimating the blurring kernel. The presented simulation as well as capture of a real life scene show how acquiring a one-shot image focused at a single plane, enable generating a de-blurred scene over an extended range in space.

In this paper, we propose a new geometric super-resolving approach that overcomes the geometric resolution reduction caused by the spatially large pixels of the detector array. The improvement process is obtained by applying an axial scanning procedure. In the scanning process, several images are captured corresponding to focus applied at several axial planes. By applying an iterative Gerchberg-Saxton-based algorithm, we managed to retrieve the phase and to reconstruct the original high-resolution image from the captured set of low-resolution images. In addition, the paper also presents a numerically efficient algorithm to compute the free space Fresnel integral.

Blurred image reconstruction using a thin phase plate combined with a post processing tool is presented. A wavelength dependent optical mask allows acquiring color channels having unique response enabling blind reconstruction of blurred images. We present deblurring of multiple-focal planes image.

We propose a new geometric super resolving approach that overcomes the geometric resolution reduction caused due to the spatially large pixels of the detection array while the improvement process is obtained by applying axial scanning and a phase retrieving procedure. In the scanning process, several images are captures corresponding to focus made on different axial plains. By applying iterative Gerchberg-Saxton based algorithm we manage to retrieve the spatial phase distribution of the optical wavefront and to reconstruct the originally geometrically high resolution image from the captured set of low resolution images.

In this paper, we generalize the method of using a 2-D moving binary random mask to overcome the geometrical resolution limitation of an imaging sensor. The spatial blurring is caused by the size of the imaging sensor pixels which yield insufficient spatial sampling. The mask is placed in an intermediate image plane and can be shifted in any direction while keeping the sensor as well as all other optical components fixed. Out of the set of images that are captured and registered, a high resolution image can be composed. In addition, this proposed approach reduces the amount of required computations and it has an improved robustness to spatial noise.

Infra-Red (IR) imaging systems are now in wide use, due to new technologies that enable manufacturing of low cost systems. Most of the IR optical materials (Germanium in particular) are very sensitive to temperature variations. This sensitivity is reflected in a temperature dependent refractive index, which in turn affects the focal length and results in strong deterioration of the image quality with temperature variations. The thermal sensitivity can be compensated using a passive all-optical phase mask. IR imaging system incorporating such simple phase mask is hereby proposed and analyzed, followed by simulation results.

In this paper we generalize and experimentally implement method of using 2-D moving binary random encoding mask positioned in the intermediate image plane in order to overcome the geometrical resolution limitation of an imaging sensor.

Present performance analysis of optical imaging systems based on results obtained with classic one-dimensional (1D) resolution targets (such as the USAF resolution chart) are significantly different than those obtained with a newly proposed 2D target [1]. We hereby prove such claim and show how the novel 2D target should be used for correct characterization of optical imaging systems in terms of resolution and contrast. We apply thereafter the consequences of these observations on the optimal design of some two-dimensional barcode structures.

A recent publication [Opt. Express16, 20540-20561 (2008)] presented a way for extending the depth of field (DOF) of imaging systems using a binary phase mask made of annular rings delivering a π-phase shift. Usually, such masks are designed with respect to some central wavelength; they will thus deliver a different phase shift for other wavelengths. This issue is reexamined in this paper, where it is shown that polychromatic masks that deliver the same phase shift over a wide range of wavelengths provide improved imaging over an extended DOF. The simulation results demonstrate the improved performance of imaging systems using such masks.

We have recently shown [Appl. Opt.51, 2739 (2012)] that performance analysis of optical imaging systems based on results obtained with classic one-dimensional (1D) resolution targets (such as the U.S. Air Force resolution target) are significantly different than those obtained with a newly proposed two-dimensional (2D) target. We hereby provide experimental evidence and show how the new 2D template can be used to correctly characterize optical imaging systems in terms of resolution and contrast. In particular, we apply the consequences of these observations to the optimal design of some 2D barcode structures.