Automated medical image segmentation is a challenging task that benefits from the use of effective image appearance models. In this paper, we compare appearance models at three regional scales for statistically characterizing image intensity near object boundaries in the context of segmentation via deformable models. The three models capture appearance in the form of regional intensity quantile functions. These distribution-based regional image descriptors are amenable to Euclidean methods such as principal component analysis, which we use to build the statistical appearance models. The first model uses two regions, the interior and exterior of the organ of interest. The second model accounts for exterior inhomogeneity by clustering on object-relative local intensity quantile functions to determine tissue-consistent regions relative to the organ boundary. The third model analyzes these image descriptors per geometrically defined local region. To evaluate the three models, we present segmentation results on bladders and prostates in CT in the context of day-to-day adaptive radiotherapy for the treatment of prostate cancer. Results show improved segmentations with more local regions, probably because smaller regions better represent local inhomogeneity in the intensity distribution near the organ boundary.

The advancing technology for automatic segmentation of medical images should be accompanied by techniques to inform the user of the local credibility of results. To the extent that this technology produces clinically acceptable segmentations for a significant fraction of cases, there is a risk that the clinician will assume every result is acceptable. In the less frequent case where segmentation fails, we are concerned that unless the user is alerted by the computer, she would still put the result to clinical use. By alerting the user to the location of a likely segmentation failure, we allow her to apply limited validation and editing resources where they are most needed. We propose an automated method to signal suspected non-credible regions of the segmentation, triggered by statistical outliers of the local image match function. We apply this test to m-rep segmentations of the bladder and prostate in CT images using a local image match computed by PCA on regional intensity quantile functions. We validate these results by correlating the non-credible regions with regions that have surface distance greater than 5.5mm to a reference segmentation for the bladder. A 6mm surface distance was used to validate the prostate results. Varying the outlier threshold level produced a receiver operating characteristic with area under the curve of 0.89 for the bladder and 0.92 for the prostate. Based on this preliminary result, our method has been able to predict local segmentation failures and shows potential for validation in an automatic segmentation pipeline.

Statistically trainable deformable-shape models (SDSMs) of anatomic objects show promise for segmentation by automatically deforming in a target image to closely match target anatomy. The purpose was to establish benchmarks for comparison with clinical studies for a particular class of SDSMs called m-reps in segmenting the prostate and bladder for IGRT/ART. Approximately 190 CT treatment images for 13 patients undergoing IGRT for prostate cancer were manually contoured by an expert. M-rep models were fit to the contours and statistically analyzed to compute a probability distribution yielding a mean and day-to-day position and shape variability for each patient's bladder and prostate. Each fitted m-rep also was registered with its corresponding image to collect, in m-rep coordinates, histograms of image intensities in predefined local regions inside and around the prostate and bladder. The histograms were processed to yield probability distributions on anatomy-relative image intensity patterns. In a leave-one-image-out experiment, mean m-rep models were applied to each treatment image for each set of patient images, where the target image was excluded from training data. The prostate m-rep was initialized in each target image via an automatically computed transformation matrix that registered the target image with the first treatment image. Due to its wide shape variability the bladder m-rep was initialized via three manually drawn contours near the base, middle and superior sections. The deformations were automatically driven by an algorithm that optimized an objective function with terms for the geometric and intensity probability distributions described above. These studies included global and regional deformations for the prostate, and global, regional and local deformations for the bladder. For the prostate the median volume overlap between m-reps and the human expert was 93% for both the prostate and bladder. The median of the average surface separation between m-rep and human segmentations over all cases was 1.1 mm for the prostate, and 1.3 mm for the bladder. This level of agreement is at voxel scale and is better than two human experts would agree. M-rep optimization mimics contouring by a human expert against which m-reps have been trained. Moreover, assuming intra-observer variabilities are similar across experts, special properties of m-reps allow the position and shape variabilities to apply to a mean model estimated by another expert, e.g., by manual contouring the planning image, without additional training. To encourage open comparison and publication of segmentation results, images and associated data sets used in this study are available to other investigators.

Based on an effective statistical segmentation methodology using a deformable medial model, a local scale deformation approach is developed to refine the global scale segmentation results within a multiscale framework. In the local scale segmentation, the probabilistic variations of locally aligned shape residuals from the global scale are learned from proper training followed by a posterior probability optimization in local regions. The resulting finer scale deformation improves the accuracy of the segmentation results, shown by experimental study on 3D CT images of the male pelvic area in day-to-day adaptive radiotherapy.

We present a novel local region approach for statistically characterizing appearance in the context of medical image segmentation via deformable models. Our appearance model reflects the inhomogeneity of tissue mixtures around the exterior of the object of interest by determining mixture-consistent local region types relative to the object boundary. The region types are formed by clustering local regional image descriptors. We partition the object boundary according to region type and apply principal component analysis on the cluster populations to acquire a statistical model of object appearance that accounts for local variability in the object exterior. We present results using this approach to segment bladders and prostates in CT in the context of day-to-day adaptive radiotherapy for prostate cancer. Results show improved fits versus those obtained with a previously developed method

For many years we have been developing a variety of methods that together would allow segmentation of 3D objects from medical images in a way reflecting knowledge of both the population of anatomic geometries sought and the population of images consistent with that geometry. To support the probability estimation methods we use to reflect this knowledge, the methods use a medial description, the m-rep, as the object representation and regional intensity quantile functions as the representation of image information in regions relative to the m-rep. Using manually segmented images to which m-reps have been fit and which contain information to allow alignment, our methods use principal geodesic analysis to estimate prior probability density, on the anatomic geometry, and they use principal component analysis to estimate a likelihood density, on the regional intensity quantile functions. They then segment automatically via posterior optimization over principal geodesic coefficients, after initialization via bones or a few contours. Each component of this methodology is briefly reviewed. Pelvic organs from multi-day populations from individual patients were segmented from CT by training a prior and a likelihood density by the methods indicated. The results are compared to human segmentations. The resulting measurements indicate that in a significant majority of cases, maximizing the log posterior objective function provides segmentations in as good or better agreement with experts than they agree with each other. Similar results are reported for other organs, other image types, and between-patient variation.

When a clinician uses an automatic method to segment a medical image, either she must accept the computer's segmentation or she must manually evaluate the quality of the segmentation and correct it as needed. This paper introduces another option: a methodology for identifying regions where the segmentation is not credible. Our method- ology identifies regions where a local geometry to image match function returns a value that is improbably poor when compared to the distribu- tion of values returned in that region for a set of training images with ac- ceptable segmentations. We validate our methodology with experiments performed on CT images of the kidney.

We present a novel histogram method for statistically characterizing the appearance of deformable models. In deformable model segmentation, appearance models measure the likelihood of an object given a target image. To determine this likelihood we compute pixel intensity quantile histograms of object-relative image regions from a weighted 3D image volume near the object boundary. We use a Gaussian model to statistically characterize the variation of histograms understood in Euclidean space via the Mallows distance. The probability of gas and bone tissue intensities are separately modeled to leverage a priori information on their expected distributions. The method is illustrated and evaluated in a segmentation study on CT images of the human left kidney. Results show improvement over a profile based appearance model and that the global maximum of the MAP estimate gives clinically acceptable segmentations in almost all of the cases studied

A statistical issue of clinical importance is intra-patient variation from day to day. We use these probability densities for segmentation of daily images by posterior optimization of deformable models. However, the information on intra- patient variation is only available after the multiple days of imaging; yet the densities are needed for segmentation on each day. Still, each patient's anatomy and image properties are distinct. We describe an approach of using sample means over the days so far to describe a Frechet mean of the patient. We assume intra-patient variation is stationary across patients, so one can pool training statistics on residues from the mean of the respective patient. The approach is applied both to principal geodesic analysis of m-rep residues describing anatomic variation and to PCA of intensity quantile residues from model-relative regions. In trials to date, application of these statistics in segmentations of male pelvic organs from CT in adaptive radiotherapy yields results competitive with human segmentations and with segmentations based fully on intra-patient statistics.

We present a novel approach to statistically characterize histograms of model-relative image regions. A multiscale model is used as an aperture to define image regions at multiple scales. We use this image description to define an appearance model for deformable model segmentation. Appearance models measure the likelihood of an object given a target image. To determine this likelihood we compute pixel intensity histograms of local model-relative image regions from a 3D image volume near the object boundary. We use a Gaussian model to statistically characterize the variation of non-parametric histograms mapped to Euclidean space using the Earth Mover’s distance. The new method is illustrated and evaluated in a deformable model segmentation study on CT images of the human bladder, prostate, and rectum. Results show improvement over a previous profile based appearance model, out-performance of statistically modeled histograms over simple histogram measurements, and advantages of regional histograms at a fixed local scale over a fixed global scale.

I present a novel parametric approach for estimating the likelihood of homogeneously textured images. I propose that the dependence between pixel features is usefully captured by estimating the joint intra-class variation of their marginal distributions. To support this claim I build a single multivariate Gaussian distribution for each class that estimates the joint variation of several marginal, nonparametric, filter response histograms. I then generalize this framework to include marginal conditional distributions of pixel intensities for use with Strong-MRF models. I demonstrate these methods on the Columbia-Utrecht database by classifying over 2800 images in all 61 texture classes. In a direct comparison with Varma & Zisserman (ECCV ’02, CVPR ’03) and Hayman (ECCV ’04) this framework is found to be more accurate and efficient.

We present a novel approach, based on local image histograms, for statistically characterizing the appearance of deformable models. In deformable model segmentation, appearance models measure the likeli- hood of an object given a target image. To determine this likelihood we compute pixel intensity histograms of local object-relative image regions from a 3D image volume near the object boundary. We use a Gaussian model to statistically characterize the variation of non-parametric his- tograms mapped to Euclidean space using the Earth Mover's Distance. The new method is illustrated and evaluated in a deformable model segmentation study on CT images of the human bladder, prostate, and rectum. Results show improvement over a previous profile based appear- ance model, out-performance of statistically modeled histograms over simple histogram measurements, and advantages of local image regions over global regions.

We face the question of how to produce a scale space of image intensities relative to a scale space of objects or other characteristic image regions filling up the image space, when both images and objects are understood to come from a population. We argue for a schema combining a multi-scale image representation with a multi-scale representation of objects or regions. The objects or regions at one scale level are produced using soft-edged apertures, which are subdivided into sub-regions. The intensities in the regions are represented using histograms. Relevant probabilities of region shape and inter-relations between region geometry and of histograms are described, and the means is given of inter-relating the intensity probabilities and geometric probabilities by producing the probabilities of intensities conditioned on geometry.

In texture classification modeling the full joint probability distribution of features is of questionable value. This paper demonstrates that marginal distributions of filter responses and marginal conditional distributions of intensity values over small neighborhoods are adequate to classify textures and can outperform methods using the joint distribution. The use of the Earth Mover’s Distance for marginal distributions is extended using PCA to build a Gaussian probability model for each class that captures the dependence between feature histograms. This framework is then generalized to include marginal conditional distributions for MRF models. These methods are demonstrated on the ColumbiaUtrecht database by classifying over 2800 images in all 61 texture classes. Results surpass those of Varma & Zisserman (CVPR ‘03) and Hayman (ECCV ’04).

Kidneys from multi-patient populations and pelvic organs from multi-day populations from a single patient were segmented by training a prior on m-rep shape and a likelihood function on regional histogram quantiles and then optimizing the posterior, finding the most probable m-rep given the image intensities. The results are compared to human segmentations and show that the log posterior objective function provides segmentations in as good or better agreement with humans than the humans agree with each other.