The standard procedure for diagnosing lung cancer involves two stages: three-dimensional (3D) computed-tomography (CT) image assessment, followed by interventional bronchoscopy. In general, the physician has no link between the 3D CT image assessment results and the follow-on bronchoscopy. Thus, the physician essentially performs bronchoscopic biopsy of suspect cancer sites blindly. We have devised a computer-based system that greatly augments the physician's vision during bronchoscopy. The system uses techniques from computer graphics and computer vision to enable detailed 3D CT procedure planning and follow-on image-guided bronchoscopy. The procedure plan is directly linked to the bronchoscope procedure, through a live registration and fusion of the 3D CT data and bronchoscopic video. During a procedure, the system provides many visual tools, fused CT-video data, and quantitative distance measures; this gives the physician considerable visual feedback on how to maneuver the bronchoscope and where to insert the biopsy needle. Central to the system is a CT-video registration technique, based on normalized mutual information. Several sets of results verify the efficacy of the registration technique. In addition, we present a series of test results for the complete system for phantoms, animals, and human lung-cancer patients. The results indicate that not only is the variation in skill level between different physicians greatly reduced by the system over the standard procedure, but that biopsy effectiveness increases.

Multidetector computed-tomography (MDCT) scanners provide large high-resolution three-dimensional (3-D) images of the chest. MDCT scanning, when used in tandem with bronchoscopy, provides a state-of-the-art approach for lung-cancer assessment. We have been building and validating a lung-cancer assessment system, which enables virtual-bronchoscopic 3-D MDCT image analysis and follow-on image-guided bronchoscopy. A suitable path planning method is needed, however, for using this system. We describe a rapid, robust method for computing a set of 3-D airway-tree paths from MDCT images. The method first defines the skeleton of a given segmented 3-D chest image and then performs a multistage refinement of the skeleton to arrive at a final tree structure. The tree consists of a series of paths and branch structural data, suitable for quantitative airway analysis and smooth virtual navigation. A comparison of the method to a previously devised path-planning approach, using a set of human MDCT images, illustrates the efficacy of the method. Results are also presented for human lung-cancer assessment and the guidance of bronchoscopy.

Bronchoscopic biopsy is often used for assisting the assessment of lung cancer. We have found in previous research that live image guidance of bronchoscopy has much potential for improving biopsy outcome. We have devised a system for this purpose. During a guided bronchoscopy procedure, our system simultaneously draws upon both the bronchoscope's video stream and the patient's 3D MDCT volume. The key data-processing step during guided bronchoscopy is the registration of the 3D MDCT data volume to the bronchoscopic video. The registration process is initialized by assuming that the bronchoscope is at a fixed viewpoint, giving a target reference video image, while the virtual-world camera inside the MDCT volume begins at an initial viewpoint that is within a reasonable vicinity of the bronchoscope's viewpoint. During registration, an optimization process searches for the optimal viewpoint to give the virtual image best matching the fixed video target. Overall, we have found that the CT-video registration technique operates robustly over a wide range of conditions, with considerable flexibility in the initial-viewpoint choice. Further, the system appears to be largely insensitive to the differences in lung capacity during the MDCT scan and during bronchoscopy. Finally, the system matches effectively in a wide range of anatomical circumstances.

Lung cancer assessment involves an initial evaluation of 3D CT image data followed by interventional bronchoscopy. The physician, with only a mental image inferred from the 3D CT data, must guide the bronchoscope through the bronchial tree to sites of interest. Unfortunately, this procedure depends heavily on the physician's ability to mentally reconstruct the 3D position of the bronchoscope within the airways. In order to assist physicians in performing biopsies of interest, we have developed a method that integrates live bronchoscopic video tracking and 3D CT registration. The proposed method is integrated into a system we have been devising for virtual-bronchoscopic analysis and guidance for lung-cancer assessment. Previously, the system relied on a method that only used registration of the live bronchoscopic video to corresponding virtual endoluminal views derived from the 3D CT data. This procedure only performs the registration at manually selected sites; it does not draw upon the motion information inherent in the bronchoscopic video. Further, the registration procedure is slow. The proposed method has the following advantages: (1) it tracks the 3D motion of the bronchoscope using the bronchoscopic video; (2) it uses the tracked 3D trajectory of the bronchoscope to assist in locating sites in the 3D CT "virtual world" to perform the registration. In addition, the method incorporates techniques to: (1) detect and exclude corrupted video frames (to help make the video tracking more robust); (2) accelerate the computation of the many 3D virtual endoluminal renderings (thus, speeding up the registration process). We have tested the integrated tracking-registration method on a human airway-tree phantom and on real human data.

Modern video based endoscopes offer physicians a wide-angle field of view (FOV) for minimally invasive procedures, Unfortunately, inherent barrel distortion prevents accurate perception of range. This makes measurement and distance judgment difficult and causes difficulties in emerging applications, such as virtual guidance of endoscopic procedures. Such distortion also arises in other wide FOV camera circumstances. This paper presents a distortion correction technique that can automatically calculate correction parameters, without precise knowledge of horizontal and vertical orientation. The method is applicable to any camera-distortion correction situation. Based on a least-squares estimation, the authors' proposed algorithm considers line fits in both FOV directions and gives a globally consistent set of expansion coefficients and an optimal image center. The method is insensitive to the initial orientation of the endoscope and provides more exhaustive FOV correction than previously proposed algorithms. The distortion-correction procedure is demonstrated for endoscopic video images of a calibration test pattern, a rubber bronchial training device, and real human circumstances. The distortion correction is also shown as a necessary component of an image-guided virtual-endoscopy system that matches endoscope images to corresponding rendered three-dimensional computed tomography views.

We present a multi-modal image registration technique that matches virtual surface renderings, derived from 3D medical data, to corresponding endoscopic video. Our efforts focus on the chest. The views of the two image sources arise from inside the major bronchial airways, as shown in 3D CT chest images and live endoscopic video. Our technique is part of a large system for two-stage assessment of lung cancer. Stage-1 involves 3D CT assessment, which gives planning data for endoscopy. Stage-2 involves live endoscopy, supplemented by CT-based guidance. The registration technique is a critical component of this assessment. The technique draws upon the concept of normalized mutual information (NMI) and works in near real time on standard PCs. An optimization procedure iteratively adjusts the virtual viewpoint rendering until it matches a given video frame's viewpoint. The optimization technique ends when the NMI is maximized. Results are given for phantom and human cases.

A system for virtual-endoscopic 3D medical image assessment and follow-on live endoscopy is described. The development and results focus on 3D CT images of the chest and lung-cancer assessment. Two stages are involved in the assessment. In Stage-1, the physician uses a series of tools for visualization and 3D central-axes analysis to build a guidance plan (a "case study"). Next, during Stage-2 endoscopy, the physician links the computer system and case study to the endoscope. The endoscopic video is registered to 3D virtual image data to give the physician "augmented reality" information to better perform the endoscopy. Results illustrate the system's utility for animal and human cases.

Transbronchial needle biopsy is a common procedure for early detection of lung cancer. In practice, accurate results are difficult to obtain, since the bronchoscopy procedure requires a blind puncture into a region hidden behind the airway walls. This paper presents an image-guided endoscopy system for procedure preplanning and for guidance during bronchoscopy. Before the bronchoscopy, a 3D CT scan is analyzed to define guidance paths through the major airways to suspect biopsy sites. During subsequent bronchoscopy, the paths give the physician step-by-step guidance to each suspect site location. At a suspect site, a virtual CT image is registered to the bronchoscopic video. Then, the predefined biopsy site, from the prior CT analysis, is rendered onto the registered video. This gives the physician a reference for performing the needle biopsy. This paper focuses on our recent experiments with this system. These experiments involve a rubber phantom model of the human airway tree and in vivo animal tests. The experiments demonstrate the promise of our approach.

Modern video-based endoscopes offer physicians a wide-angle held of view for minimally-invasive procedures. Unfortunately, inherent barrel distortion prevents accurate perception of range. This makes measurement and distance judgment difficult and causes difficulties in emerging applications, such as 3D medical-image registration. Such distortion also arises in other wide field-of-view camera circumstances. This paper presents a distortion-correction technique that can automatically calculate correction parameters, without precise knowledge of horizontal and vertical-orientation. The method is applicable to any camera-distortion correction situation. Based on a least-squares estimation, our proposed algorithm considers line fits in both field-of-view directions and global consistency that gives the optimal image center and expansion coefficients. The method is insensitive to the initial orientation of the endoscope and provides more exhaustive field-of-view correction than previously proposed algorithms. The distortion-correction procedure is demonstrated for endoscopic video images of a calibration test pattern, a rubber bronchial training device, and real human circumstances. The distortion correction is also shown as a necessary component of an image-guided virtual-endoscopy system that matches endoscope images to corresponding rendered 3D CT views.

To improve the care of lung-cancer patients, rye are devising a diagnostic paradigm that ties together three-dimensional (3D) high-resolution computed-tomographic (CT) imaging and bronchoscopy. The system expands upon the new concept of virtual endoscopy that has seen recent application to the chest, colon, and other anatomical regions. Our approach applies computer-graphics and image-processing tools to the analysis of 3D CT chest images and complementary bronchoscopic video. It assumes a two-stage assessment of a lung-cancer patient. During Stage 1 (CT assessment), the physician interacts with a number of visual and quantitative tools to evaluate the patient's "virtual anatomy" (3D CT scan). Automatic analysis gives navigation paths through major airways and to preselected suspect sites. These paths provide useful guidance during Stage-1 CT assessment. While interacting with these paths and other software tools, the user builds a multimedia Case Study, capturing telling snapshot views, movies, and quantitative data. The Case Study contains a report on the CT scan and also provides planning information for subsequent bronchoscopic evaluation. During Stage 2 (bronchoscopy), the physician uses (1) the original CT data. (2) software graphical tools, (3) the Case Study, and (4) a standard bronchoscopy suite to have an augmented vision for bronchoscopic assessment and treatment. To use the two data sources (CT and bronchoscopic video) simultaneously, they must be registered. We perform this registration using both manual interaction and an automated matching approach based on mutual information. We demonstrate our overall progress to date using human CT cases and CT-video from a bronchoscopy-training device.

For original paper see ibid., vol. 33, no. 4, p. 1355-61 (1996). Comments on the equations of /spl sigma//sub x//sup 2/, /spl sigma//sub y//sup 2/ and /spl sigma//sub xy/. The derivations of these variances of the noises in r/sub x/, r/sub y/ and r/sub xy/ are not given. A reply is published.

Tactically maneuvering targets are difficult to track since acceleration cannot be observed directly and the accelerations are induced by human control or an autonomous guidance system; therefore they are not subject to deterministic models. A common tracking system is the two-state Kalman Filter with a Singer maneuver model where the second order statistics of acceleration is the same as a first order Markov process. The Singer model assumes a uniform probability distribution on the target's acceleration which is independent of the x and y direction. In practice, it is expected that targets have constant forward speed and an acceleration vector normal to the velocity vector, a condition not present in the Singer model. This paper extends the work of Singer by presenting a maneuver model which assumes constant forward speed and a probability distribution on the targets turn-rate. Details of the model are presented along with sample simulation results.<>

Premature and newborn infants often have underdeveloped lungs and, as a result, benefit from additional oxygen. Unfortunately, too much oxygen risks inducing blindness as well as other toxicity phenomena. Since the infant's oxygen uptake and delivery vary widely and are constantly changing, automatic control of the supplied oxygen is desirable. Many of the issues and technologies of tactical control are applicable to this problem: real-time control, expert system knowledge, multi-sensor corroboration and data fusion, failure identification and artifact rejection, etc. This paper explores these issues as well as the results of current PSU research in neonate oxygen control

This paper compares two performance indices for computing optimal observer paths for the bearings-only source localization and tracking problem, for constant velocity sources. Previous work on this problem is based on maximizing the determinant of the Fisher information matrix (FIM) of the estimation problem. This paper considers minimizing the trace of a weighted sum of the Cramer-Rao lower bound (CRLB) of current or future source position errors, and source velocity errors. Quasi-Newton optimization is used to compare optimal observer paths, given three distinct goals: minimizing current position error, velocity error, and future position error. Significant differences in optimal paths are observed, and the CRLB trace is found to yield smaller estimation ambiguity

This paper compares two performance indices for computing optimal observer paths for the bearings-only source constant velocity sources. The problem is based on maximizing the determinant of the Fisher information matrix (FIM) of the estimation problem. It considers minimizing the trace of a weighted sum of the Cramer-Rao lower bound (CRLB) of current source position error. Quasi-Newton optimization is used to compare optimal observer paths, given the goal of minimizing current position error. Significant differences in optimal paths are observed, and the CRLB trace is found to yield smaller range error.< >