Statistically trainable deformable-shape models (SDSMs) of anatomic objects show promise for segmentation by automatically deforming in a target image to closely match target anatomy. The purpose was to establish benchmarks for comparison with clinical studies for a particular class of SDSMs called m-reps in segmenting the prostate and bladder for IGRT/ART. Approximately 190 CT treatment images for 13 patients undergoing IGRT for prostate cancer were manually contoured by an expert. M-rep models were fit to the contours and statistically analyzed to compute a probability distribution yielding a mean and day-to-day position and shape variability for each patient's bladder and prostate. Each fitted m-rep also was registered with its corresponding image to collect, in m-rep coordinates, histograms of image intensities in predefined local regions inside and around the prostate and bladder. The histograms were processed to yield probability distributions on anatomy-relative image intensity patterns. In a leave-one-image-out experiment, mean m-rep models were applied to each treatment image for each set of patient images, where the target image was excluded from training data. The prostate m-rep was initialized in each target image via an automatically computed transformation matrix that registered the target image with the first treatment image. Due to its wide shape variability the bladder m-rep was initialized via three manually drawn contours near the base, middle and superior sections. The deformations were automatically driven by an algorithm that optimized an objective function with terms for the geometric and intensity probability distributions described above. These studies included global and regional deformations for the prostate, and global, regional and local deformations for the bladder. For the prostate the median volume overlap between m-reps and the human expert was 93% for both the prostate and bladder. The median of the average surface separation between m-rep and human segmentations over all cases was 1.1 mm for the prostate, and 1.3 mm for the bladder. This level of agreement is at voxel scale and is better than two human experts would agree. M-rep optimization mimics contouring by a human expert against which m-reps have been trained. Moreover, assuming intra-observer variabilities are similar across experts, special properties of m-reps allow the position and shape variabilities to apply to a mean model estimated by another expert, e.g., by manual contouring the planning image, without additional training. To encourage open comparison and publication of segmentation results, images and associated data sets used in this study are available to other investigators.

Purpose: Explicit deformable shape models (DSMs) can be used in a Bayesian statistical framework to provide a priori information for posterior optimization to match the DSM against a target image for automatic segmentation. In this approach a DSM is initialized in the target image and undergoes a series of deformations to closely match the target object. Deformation is driven by optimizing an objective function with terms for geometric typicality (prior) and model‐to‐image match (likelihood). The purpose of this work was to develop strategy, methodology, and tools for constructing the geometric prior and intensity likelihood for a particular form of DSM called m‐reps. Method and Materials: Geometric truth is defined for an object of interest by a statistically significant collection of expert human segmentations of training images. M‐reps are fit to the human drawn contours by minimizing the distance between the surfaces of the m‐rep and the contours under added conditions that lead to positional correspondence across training cases. The geometry of the resulting set of training m‐reps is analyzed in non‐Euclidean space using an approach called principal geodesic analysis (PGA) to yield a set of eigenmodes that define the geometric prior. The intensity likelihood is constructed by registering each training m‐rep with the corresponding gray scale image and collecting regional intensity information that is statistically characterized over all training cases. The intensity information can be in several forms including linear profiles and regional histograms. Results: PGA produces modes that include natural deformations such as local twisting, bending, bulging, and constricting. Unlike analysis in Euclidean space, improper shapes are avoided. The form of the intensity prior can be customized to each object of interest for optimal performance. Conclusion: These methods are powerful, robust and generalizable to other DSMs. Conflict of Interest: The presenting author has a financial interest in Morphormics, Inc.

PURPOSE:A controlled observer study was conducted to compare a method for automatic image segmentation with conventional user-guided segmentation of right and left kidneys from planning computerized tomographic (CT) images. METHODS AND MATERIALS:Deformable shape models called m-reps were used to automatically segment right and left kidneys from 12 target CT images, and the results were compared with careful manual segmentations performed by two human experts. M-rep models were trained based on manual segmentations from a collection of images that did not include the targets. Segmentation using m-reps began with interactive initialization to position the kidney model over the target kidney in the image data. Fully automatic segmentation proceeded through two stages at successively smaller spatial scales. At the first stage, a global similarity transformation of the kidney model was computed to position the model closer to the target kidney. The similarity transformation was followed by large-scale deformations based on principal geodesic analysis (PGA). During the second stage, the medial atoms comprising the m-rep model were deformed one by one. This procedure was iterated until no changes were observed. The transformations and deformations at both stages were driven by optimizing an objective function with two terms. One term penalized the currently deformed m-rep by an amount proportional to its deviation from the mean m-rep derived from PGA of the training segmentations. The second term computed a model-to-image match term based on the goodness of match of the trained intensity template for the currently deformed m-rep with the corresponding intensity data in the target image. Human and m-rep segmentations were compared using quantitative metrics provided in a toolset called Valmet. Metrics reported in this article include (1) percent volume overlap; (2) mean surface distance between two segmentations; and (3) maximum surface separation (Hausdorff distance). RESULTS:Averaged over all kidneys the mean surface separation was 0.12 cm, the mean Hausdorff distance was 0.99 cm, and the mean volume overlap for human segmentations was 88.8%. Between human and m-rep segmentations the mean surface separation was 0.18-0.19 cm, the mean Hausdorff distance was 1.14-1.25 cm, and the mean volume overlap was 82-83%. CONCLUSIONS:Overall in this study, the best m-rep kidney segmentations were at least as good as careful manual slice-by-slice segmentations performed by two experienced humans, and the worst performance was no worse than typical segmentations from our clinical setting. The mean surface separations for human-m-rep segmentations were slightly larger than for human-human segmentations but still in the subvoxel range, and volume overlap and maximum surface separation were slightly better for human-human comparisons. These results were expected because of experimental factors that favored comparison of the human-human segmentations. In particular, m-rep agreement with humans appears to have been limited largely by fundamental differences between manual slice-by-slice and true three-dimensional segmentation, imaging artifacts, image voxel dimensions, and the use of an m-rep model that produced a smooth surface across the renal pelvis.

Deformable shape models (DSMs) comprise a general approach that shows great promise for automatic image segmentation. Published studies by others and our own research results strongly suggest that segmentation of a normal or near-normal object from 3D medical images will be most successful when the DSM approach uses (1) knowledge of the geometry of not only the target anatomic object but also the ensemble of objects providing context for the target object and (2) knowledge of the image intensities to be expected relative to the geometry of the target and contextual objects. The segmentation will be most efficient when the deformation operates at multiple object-related scales and uses deformations that include not just local translations but the biologically important transformations of bending and twisting, i.e., local rotation, and local magnification. In computer vision an important class of DSM methods uses explicit geometric models in a Bayesian statistical framework to provide a priori information used in posterior optimization to match the DSM against a target image. In this approach a DSM of the object to be segmented is placed in the target image data and undergoes a series of rigid and nonrigid transformations that deform the model to closely match the target object. The deformation process is driven by optimizing an objective function that has terms for the geometric typicality and model-to-image match for each instance of the deformed model. The success of this approach depends strongly on the object representation, i.e., the structural details and parameter set for the DSM, which in turn determines the analytic form of the objective function. This paper describes a form of DSM called m-reps that has or allows these properties, and a method of segmentation consisting of large to small scale posterior optimization of m-reps. Segmentation by deformable m-reps, together with the appropriate data representations, visualizations, and user interface, has been implemented in software that accomplishes 3D segmentations in a few minutes. Software for building and training models has also been developed. The methods underlying this software and its abilities are the subject of this paper.

We present a novel approach, clustering on local image profiles, for statistically characterizing image intensity in object boundary regions. In deformable model segmentation, a driving consideration is the geometry to image match, the degree to which the target image conforms to some template within the object boundary regions. The template should account for variation over a training set and yet be specific enough to drive an optimization to a desirable result. Using clustering, a template can be built that is optimal over the training data in the metric used, such as normalized correlation. We present a method that first determines local cross-boundary image profile types in the space of training data and then builds a template of optimal types. Also presented are the results of a study using this approach on the human kidney in the context of medial representation deformable model segmentation. The results show an improvement in the automatic segmentations using the cluster template, over a previously built template.

The purpose of this study was to evaluate m-rep deformable models for automatic segmentation of multi-object soft-tissue complexes in low contrast images with application to bladder, rectum and prostate in planning and treatment CT images. M-rep deformation is guided by knowledge of object and inter-object geometry and image intensity patterns gained through a statistical training procedure. The method is broadly applicable but is tested here for adaptive radiotherapy for prostate cancer, where the segmentation is applied to CT images acquired in the treatment room over the course of radiotherapy. M-reps *S Pizer et al, Deformable M-Reps for 3D Medical Image Segmentation, Int J Comp Vision, 55: 85–106 (2003) are deformable models that describe anatomical objects in terms of a mean shape and local broadeningselongation, twistingsbending, and displacement of volume elements of the object(s) forming the mean. M-reps also model non-rigid between-object and within-object geometry such as the relationship between adjacent non-interpenetrating structures. Principal modes of shape variation computed in non-Euclidean space provide statistics of variability of the object(s) within a training population. Separately, image intensity statistics are generated in m-rep object-relative coordinates from the same training images. The mean model is deformed in a target image by successive optimization to yield the most probable segmentation given the image data. The objective function reflects both the shape statistics and the intensity statistics produced at training. The deformation proceeds from the multi-object complex as a whole, to single objects starting from inter-object predictions, to intra-object volume sections. For the male pelvis the supra-pubic bones are segmented first, followed by the bladder/rectum/prostate complex, and finally the individual soft tissue structures. In this study of the applicability of m-reps for adaptive therapy, the same-patient “mean” model and image intensity training were derived from the planning image. Also the shape statistics in this study describe the transformation between the planning image model and the treatment images. M-rep segmentations obtained in this study were compared to expert human segmentations by statistical analysis of metrics including volume overlap, mean surface separation, maximum surface separation, and separation-distance histogram quartiles. Clinically acceptable, non-interpenetrating segmentations of the prostate and other soft tissues were obtained. Automatic m-rep segmentation works well for adaptive radiotherapy for prostate cancer. The effectiveness of m-reps is attributed to several important properties: 1) the ability to represent solid volume geometry; 2) geometric modeling that reflects both local shape and local inter-regional relations; 3) the inclusion of statistics describing geometric properties of individual objects and object groups and of inter-relationships between these objects; 4) the inclusion of statistical aspects of intensity patterns in object-relative geometry; and 5) the multiscale nature both of training and of segmentation of target images. The next step is generalization to the multi-patient situation. This will involve more stages of statistical training, more stages in the model deformation process, and greater attention to the variability of relationships between the various structures comprising the multi-object m-rep model.