A recent trend in computer vision is to combine the census cost function with a TV-L1 energy minimization scheme. Although this combination is known for its robust performance in computer vision applications, it has not been introduced to 3D medical image registration yet. Addressing pulmonary motion estimation in 4D (3D+t) CT images, we propose incorporating the census cost function into a 3D implementation of the 'duality-based approach for realtime TV-L1 optical flow' for the task of lung CT registration. The performance of the proposed algorithm is evaluated on the DIR-lab benchmark and compared to state-of-the-art approaches in this field. Results highlight the potential of the census cost function for accurate pulmonary motion estimation in particular, and 3D medical image registration in general.

In 2004, Brox et al. described how to minimize an energy functional for dense 2D optical flow estimation that enforces both intensity and gradient constancy.This paper presents a novel variant of their method, in which the census cost function is utilized in the data term instead of absolute intensity differences. The algorithm is applied to the task of pulmonary motion estimation in 3D computed tomography (CT) image sequences. The performance evaluation is based on DIR-lab benchmark data for lung CT registration. Results show that the presented algorithm can compete with current state-of-the-art methods in regards to both registration accuracy and run-time.

Scan-line optimization via cost accumulation has become very popular for stereo estimation in computer vision applications and is often combined with a semi-global cost integration strategy, known as SGM.This paper introduces this combination as a general and effective optimization technique. It is the first time that this concept is applied to 3D medical image registration.The presented algorithm, SGM-3D, employs a coarse-to-fine strategy and reduces the search space dimension for consecutive pyramid levels by a fixed linear rate. This allows it to handle large displacements to an extent that is required for clinical applications in high dimensional data.SGM-3D is evaluated in context of pulmonary motion analysis on the recently extended DIR-lab benchmark that provides ten 4D computed tomography (CT) image data sets, as well as ten challenging 3D CT scan pairs from the COPDgene study archive. Results show that both registration errors as well as run-time performance are very competitive with current state-of-the-art methods.

Dense and robust optical flow estimation is still a major challenge in low-level computer vision. In recent years, mainly variational methods contributed to the progress in this field. One reason for their success is their suitability to be embedded into hierarchical schemes, which makes them capable of handling large pixel displacements. Matching-based regularization techniques, like dynamic programming or belief propagation concepts, can also lead to accurate optical flow fields. However, results are limited to short- or mid-scale optical flow vectors, because these techniques are usually not combined with coarse-to-fine strategies. This paper introduces fSGM, a novel algorithm that is based on scan-line dynamic programming. It uses the cost integration strategy of semi-global matching, a concept well known in the area of stereo matching. The major novelty of fSGM is that it embeds the scan-line dynamic programming approach into a hierarchical scheme, which allows it to handle large pixel displacements with an accuracy comparable to variational methods. We prove the exceptional performance of fSGM by comparing it to current state-of-the-art methods on the KITTI Vision Benchmark Suite.

Semi-global matching (SGM) is a technique of choice for dense stereo estimation in current industrial driver-assistance systems due to its real-time processing capability and its convincing performance. In this paper we introduce iSGM as a new cost integration concept for semi-global matching. In iSGM, accumulated costs are iteratively evaluated and intermediate disparity results serve as input to generate semi-global distance maps. This novel data structure supports fast analysis of spatial disparity information and allows for reliable search space reduction in consecutive cost accumulation. As a consequence horizontal costs are stabilized which improves the robustness of the matching result. We demonstrate the superiority of this iterative integration concept against a standard configuration of semi-global matching and compare our results to current state-of-the-art methods on the KITTI Vision Benchmark Suite.

This paper presents a novel way of combining dense stereo and motion analysis for the purpose of mid-level scene segmentation and object tracking. The input is video data that addresses long-range stereo analysis, as typical when recording traffic scenes from a mobile platform. The task is to identify shapes of traffic-relevant objects without aiming at object classification at the considered stage. We analyse disparity dynamics in recorded scenes for solving this task. Statistical shape models are generated over subsequent frames. Shape correspondences are established by using a similarity measure based on set theory. The motion of detected shapes (frame to frame) is compensated by using a dense motion field as produced by a real-time optical flow algorithm. Experimental results show the quality of the proposed method which is fairly simple to implement.

In this study we examine three road-modeling methods, which we evaluate on seven stereo matching algorithms. The road-modeling methods we consider are a B-spline modeling technique based on region-growing and two versions of the popular v-disparity approach. The used stereo algorithms are variations or different parameterizations of belief propagation, graph cut and semi-global matching. We examine the performance of the possible combinations of modeling methods and stereo algorithms by comparing the deviation towards a reference road profile. Two evaluation sequences are used, of which one features switched on windscreen wipers that are visible in the the recorded imagery. Our findings are that the examined B-spline modeling method provides the best results in most cases. In terms of the modeled distance, belief propagation is the most suitable stereo matching method, which also performs best with the wiper sequence. Semi global matching achieves smaller model deviations, but with drastically reduced modeled distances.

The paper considers semi-global stereo matching in the context of vision-based driver assistance systems. The need for real-time performance in this field requires a design change of the originally proposed method to run on current hardware. This paper proposes such a new design; the novel strategy first generates a disparity map from half-resolution input images. The result is then used as prior to restrict the disparity search space for full-resolution computation. This approach is compared to an SGM strategy as employed currently in a state-of-the-art real-time FPGA solution. Furthermore, trinocular stereo evaluation is performed on ten real-world traffic sequences with a total of 4,000 trinocular frames. An extension to the original evaluation methodology is proposed to resolve ambiguities and to incorporate disparity density in a statistically meaningful way. Evaluation results indicate that the novel SGM method is up to 40% faster when compared to the previous strategy. It returns denser disparity maps, and is also more accurate on evaluated traffic scenes.

Prediction error analysis is a known method for evaluating the performance of optic flow algorithms. This paper applies this method to long image sequences with a special focus on opportunities when using metronome sequences. Two top-performing algorithms are discussed, known as TV-L1 and BBPW.

Evaluation of stereo-analysis algorithms is usually done by analysing the performance of stereo matchers on data sets with available ground truth. The trade-off between precise results, obtained with this sort of evaluation, and the limited amount (in both, quantity and diversity) of data sets, needs to be considered if the algorithms are required to analyse real-world environments. This chapter discusses a technique to objectively evaluate the performance of stereo-analysis algorithms using real-world image sequences. The lack of ground truth is tackled by incorporating an extra camera into a multi-view stereo camera system. The relatively simple hardware set-up of the proposed technique can easily be reproduced for specific applications.

The paper evaluates three categories of similarity measures: ordering-based (census), gradient-based, and illumination-based cost functions. The performance of those functions is evaluated especially with respect to illumination changes using two different sets of data, also including real world driving sequences of hundreds of stereo frames with strong illumination differences. The overall result is that there are cost functions in all three categories that can perform well on a quantitative and qualitative level. This leads to the assumption that those cost functions are in fact closely related at a qualitative level, and we provide our explanation.

Semi-global matching is a popular choice for applications where dense and robust stereo estimation is required and real-time performance is crucial. It therefore plays an important role in vision-based driver assistance systems. The strength of the algorithm comes from the integration of multiple 1D energy paths which are minimized along eight different directions across the image domain. The contribution of this paper is twofold. First, a thorough evaluation of stereo matching quality is performed when the number of accumulation paths is reduced. Second, an alteration of semi-global matching is proposed that operates only on half of the image domain without losing disparity resolution. The evaluation is performed on four real-world driving sequences of 400 frames each, as well as on 396 frames of a synthetic image sequence where sub-pixel accurate ground truth is available. Results indicate that a reduction of accumulation paths is a very good option to improve the run-time performance without losing significant quality, even on sub-pixel level. Furthermore, operating semiglobal matching only on half the image yields almost identical results to the corresponding full path integration. This approach yields the potential to further speed up the runtime and could also be exploited for other alterations of the algorithm.

Using gradient information for a pixel-based cost function for stereo matching has lacked adequate attention in the literature. This paper provides experimental evidence to show that the gradient as a data descriptor outperforms other pixel-based functions such as absolute differences and the Birchfield and Tomasi cost functions. The cost functions are tested against stereo image datasets where ground truth data is available. Furthermore, analysing the effect of the cost functions when exposure and illumination settings are different between the left and right camera is analysed. Not only has the performance of the cost functions been analysed, but also analysis into “why” one cost function is better than another. The analysis tests the global and spacial optimality of the cost function, showing that the gradient information returns stronger minima than the other two. These results are aimed at future research towards the design of a new smoothness prior that also depends on the characteristics of the employed cost function. This paper shows that the gradient is a simple, yet powerful, data descriptor that shows robustness to illumination and exposure differences, but is often overlooked by the stereo community.

The paper proposes the prediction of stereo matching performance based on analyzing the given stereo data (and not based on test runs of stereo matching algorithms). For justifying our approach we compare results obtained by prediction error analysis (for different stereo matching algorithms) with three different data evaluation techniques: a count of SIFT matches, a mismatch count between census transform features, and the quality of dense optical flow fields based on a total-variation energy minimization. The paper shows that there are reasonable indications that such measures, quantifying matches of features or image regions, correlate with stereo performance to some degree. This study on data evaluation is initiating a new direction of research, and it concludes with the suggestion of studying further measures or more data for the ultimate goal of supporting an adaptive optimization or selection of stereo matching techniques with respect to given image data.

Today's stereo vision algorithms and computing technology allow real-time 3D data analysis, for example for driver assistance systems. A recently developed Semi-Global Matching (SGM) approach by H. Hirschmüller became a popular choice due to performance and robustness. This paper evaluates different parameter settings for SGM, and its main contribution consists in suggesting to include a second order prior into the smoothness term of the energy function. It also proposes and tests a new cost function for SGM. Furthermore, some preprocessing (edge images) proved to be of great value for improving SGM stereo results on real-world sequences, as previously already shown by S. Guan and R. Klette for belief propagation. There is also a performance gain for engineered stereo data (e.g.) as currently used on the Middlebury stereo website. However, the fact that results are not as impressive as on the .enpeda.. sequences indicates that optimizing for engineered data does not neccessarily improve real world stereo data analysis.

Curvature is a frequently used property in two-dimensional (2D) shape analysis, directly or for derived features such as corners or convex and concave arcs. This paper presents curvature estimators which follow approaches in differential geometry. Digital-straight segment approximation (as known from digital geometry) is used in those estimators. Results of multigrid experiments are evaluated leading to a comparative performance analysis of several curvature estimators

We introduce a novel technique that allows for an automatic quantification of MR DTI parameters along arbitrarily oriented fiber bundles. Most previous methods require either a manual placement of ROIs, are limited to single fiber tracts, or are limited to bundles which are perpendicular to one of the three image planes. Thus, the quantification process is made much more time-efficient and robust by our new approach. We compare our technique with a manual quantification of an expert and show the similarity of the results. Furthermore, we demonstrate how to visualize the parameters at a certain position of the fiber bundle so that areas of interest can easily be examined.

The paper starts with presenting three curvature estimators which follow definitions (approaches) in differential geometry. Digital-straight segment (DSS) approximation is used in those estimators, we point to problems caused by this approach, and propose simple ways for eliminating those problems. The paper then informs about multigrid analysis experiments, where all estimators appear to be multigrid convergent when digitizing an ellipse. The paper also applies these estimators for corner detection and compares their performance with a recently published heuristic corner-detection approach by means of multigrid analysis. Experiments indicate that corner detectors (based on curvature estimation) perform about as good as the heuristic method for large grid resolutions, and one detector might be even superior.

This article explains a new method for the estimation of curvature of plane curves and compares it with a method which has been presented in (2). Both methods are based on global approximations of tangents by digital straight line segments. Experimental studies show that a replacement of global by local approximation results in errors which, in contrast to the global approximation, converge to constants > 0. We also apply the new global method for curvature estimation of curves to surface curvature estimation, and discuss a method for estimating mean curvature of surfaces which is based on Meusnier's theorem.