Locating bronchial lesions appearing in multimodal bronchoscopic videos collected during a patient airway exam has the potential to be a major minimally invasive method for the early detection of cancerous lesions. Unfortunately, no true multimodal method exists for automatically detecting bronchial lesions arising in such video data. Multimodal lesion detection requires a method to directly correlate different real-world data sources at the same virtual space camera pose. We explore the use of an enhanced 4D Gaussian splatting method for synthesizing endoscopic video, particularly at suspect lesion sites, for two separate objectives. The first objective is to synthesize endoscopic video from novel camera poses using a single data source. The second is to extend the video synthesis to multiple data sources, thereby enabling the direct video comparisons we desire for multimodal lesion detection. To this end, we propose an enhanced endoscopic video synthesis method that improves upon the popular Deform3DGS approach developed by Yang et al. by improving the approach's initial 3D point cloud and estimated depth map inputs. Our proposed method further incorporates information from a reference CT-based chest model of the endoluminal airway anatomy scanned by the endoscopic video sources into producing a more geometrically accurate initial point cloud. We also introduce a custom Monodepth2 depth estimation network, trained specifically on multimodal bronchoscopic video data, to provide improved estimated depth maps. Results using multimodal bronchoscopic video data from lung cancer patients show that the proposed method's performance exceeds that of the Deform3DGS approach for our goal of synthesizing single-mode endoscopic video at unseen poses. It also provides a feasible means of performing our other goal of directly comparing multimodal endoscopic video.

The successful management of lung cancer patients requires timely and accurate disease diagnosis, through peripheral lesion analysis, and effective disease staging, through comprehensive central-chest lymph node staging. The state-of-the-art approach for performing follow-on diagnosis and staging procedures entails minimally invasive bronchoscopy. Because bronchoscopy is challenging, the medical device community has embarked on a massive ongoing effort to create assisted bronchoscopy systems. Unfortunately, current systems suffer from many drawbacks: they do not enable sufficiently accurate peripheral lesion diagnosis; they offer no guidance for nodal staging across both lungs; and they offer no guidance for supplemental devices typically needed during a procedure; they tend to subject a patient to significant radiation from supplemental imaging devices and demand long procedure times. As a step toward mitigating these drawbacks, we have devised two complementary turnkey subsystems for peripheral lesion analysis and central-chest lymph node staging, both of which had been previously tested in phantom, animal, and human feasibility studies. We now present a prospective patient study for the combined system for lung cancer diagnosis and staging within the context of the live clinical workflow. We also compare the results of this study to a historical controls patient cohort performed at our University's medical center. Results indicate that our guidance system has the potential to enable far faster peripheral lesion bronchoscopies and nodal staging bronchoscopies, in comparison to the historical cohort. Also, the guidance system can potentially enable more efficient and accurate use of radial endobronchial ultrasound for peripheral lesions, without needing to resort to a radiation-intensive confirmational imaging modality, while also enabling the examination of substantially more nodal sites. With a mean joint procedure time under 10 minutes, the system could help realize more efficient and effective lung cancer diagnosis/staging bronchoscopies.

Early detection of lung cancer is crucial as it significantly improves survival rates by facilitating timely and effective treatment. Lung cancer often begins as bronchial lesions developing along the airway walls. Bronchoscopy is the minimally invasive method of choice for detecting such lesions. Currently, three complementary bronchoscopic video modalities have been utilized for this purpose: white-light bronchoscopy (WLB), narrow-band imaging (NBI), and autofluorescence bronchoscopy (AFB). Unfortunately, current practice forces the clinician to manually examine each video source and later interactively correlate the results of these exams to make final lesion decisions. Because of the lack of effective tools for multimodal endoscopic video analysis, this proves to be an extremely time-consuming, error-prone process, making it impractical for common clinical use. To address this problem, we propose a multimodal video analysis and synchronization system that enables efficient analysis of multimodal bronchoscopic videos for early cancer lesion detection. The system provides methods for planning and guiding a straightforward multimodal airway exam through the major airways. Subsequent video processing methods then draw on deep-learning-based techniques to identify candidate single-mode bronchial lesions. Next, a synchronization/registration pipeline registers all bronchoscopic video data to a reference 3D airway tree model derived from a patient's X-ray computed tomography (CT) scan. This finally facilitates interactive graphical visualization and interaction with all processed multimodal data. Results with lung cancer patient studies indicate the system's promise for efficient, effective video analysis and lesion detection.

PurposeLung cancer remains the leading cause of cancer death. This has brought about a critical need for managing peripheral regions of interest (ROIs) in the lungs, be it for cancer diagnosis, staging, or treatment. The state-of-the-art approach for assessing peripheral ROIs involves bronchoscopy. To perform the procedure, the physician first navigates the bronchoscope to a preplanned airway, aided by an assisted bronchoscopy system. They then confirm an ROI's specific location and perform the requisite clinical task. Many ROIs, however, are extraluminal and invisible to the bronchoscope's field of view. For such ROIs, current practice dictates using a supplemental imaging method, such as fluoroscopy, cone-beam computed tomography (CT), or radial endobronchial ultrasound (R-EBUS), to gather additional ROI location information. Unfortunately, fluoroscopy and cone-beam CT require substantial radiation and lengthen procedure time. As an alternative, R-EBUS is a safer real-time option involving no radiation. Regrettably, existing assisted bronchoscopy systems offer no guidance for R-EBUS confirmation, forcing the physician to resort to an unguided guess-and-check approach for R-EBUS probe placement—an approach that can produce R-EBUS placement errors exceeding 30 deg, an error that can result in missing many ROIs. Thus, because of physician skill variations, biopsy success rates using R-EBUS for ROI confirmation have varied greatly from 31% to 80%. This situation obliges the physician to turn to a radiation-based modality to gather sufficient information for ROI confirmation. We propose a two-phase registration method that provides guidance for R-EBUS probe placement.ApproachAfter the physician navigates the bronchoscope to the airway near a target ROI, the two-phase registration method begins by registering a virtual bronchoscope to the real bronchoscope. A virtual 3D R-EBUS probe model is then registered to the real R-EBUS probe shape depicted in the bronchoscopic video using an iterative region-based alignment method drawing on a level-set-based optimization. This synchronizes the guidance system to the target ROI site. The physician can now perform the R-EBUS scan to confirm the ROI.ResultsWe validated the method's efficacy for localizing extraluminal ROIs with a series of three studies. First, for a controlled phantom study, we observed that the mean accumulated position and direction errors (accounting for both registration phases) were 1.94 mm and 3.74 deg (equivalent to 1.30 mm position error for a 20 mm biopsy needle), respectively. Next, for a live animal study, these errors were 2.81 mm and 4.79 deg (2.41 mm biopsy needle error), respectively. For 100% of the ROIs considered in these two studies, the method enabled visualization of an ROI via R-EBUS in under 3 min per ROI. Finally, initial operating-room tests on lung cancer patients indicated the method's efficacy, functionality, efficiency, and safety under standard clinical conditions.ConclusionsThe method offers a quick, low-cost, radiation-free approach for examining peripheral extraluminal ROIs using R-EBUS. Although our studies focused on R-EBUS as the supplemental working channel instrument, the proposed method has general applicability to any clinical bronchoscopic task requiring a working channel instrument. Thus, the method has the potential to improve the efficiency and efficacy of bronchoscopic procedures for lung cancer patients.

The formation of bronchial lesions along the airway walls (mucosa) is known to be a potential indicator of early lung cancer. White-light bronchoscopy (WLB) has long been the standard minimally invasive modality for identifying bronchial lesions. To do this, the physician performs a procedure known as bronchoscopy, examining the major airways to interactively locate suspect lesion sites. Unfortunately, past clinical studies have established that such an exam demands a time-consuming, careful inspection of the lengthy incoming WLB video stream. Because suspect lesions are typically distinguished from normal background regions by subtle differences, many lesions are easily missed in the video stream, resulting in lesion detection rates as low as 29% in past clinical studies. The recent introduction of high-definition (HD) WLB offers the potential for more detailed airway wall imaging, thereby offering a potentially more informative data source for identifying lesions. Nevertheless, the physician still must rely on interactive inspection to identify lesions. We consider deep learning approaches for automating the process of analyzing WLB video. In particular, we develop an HD WLB ground truth dataset and apply this dataset to a series of deep learning models to the problem of automatic bronchial lesion detection and segmentation. In our study, the best model achieved 97% detection accuracy and a Dice score of 0.54 for lesion segmentation.

Early detection of lung cancer allows for more effective treatment and helps increase the likelihood of patient survival. This fact has inspired the search for biomarkers that can help indicate disease development and cancer risk. One important minimally invasive method for identifying potential biomarkers entails performing an airway exam using bronchoscopy. More specifically, autofluorescence bronchoscopy (AFB) is notable for its high sensitivity in detecting candidate early cancer lesions along the airway wall. The task of performing an AFB airway exam to identify such lesions, however, proves to be very tedious, error prone and overly dependent on physician skill. This is due to the lack of sufficient tools to facilitate efficient, accurate analysis of the airway exam's video stream. We propose an integrated interactive system for autofluorescence bronchoscopy. The system takes a patient's three-dimensional (3D) chest computed tomography (CT) scan and a live bronchoscopy video stream as inputs and provides the following capabilities: 1) guidance assistance for performing the airway exam; 2) automatic video analysis to produce real-time detection and segmentation of candidate lesions; 3) subsequent lesion tracking over the video sequence to identify key frames that denote the most representative locations of detected lesions; 4) visualization and interaction tools to view lesion detection outcomes and to make final lesion confirmation decisions; 5) graphical tools for showing a detected lesion's precise anatomical location within the 3D airway tree. Through these capabilities, the system has the capacity to deliver a comprehensive assessment of suspect lesions throughout an AFB airway exam. Utilizing the exam videos and CT scans from lung cancer patients, we demonstrate the potential of our system for real-time, systematic analysis of a patient's major airways.

Narrow-band imaging (NBI), a relatively new bronchoscopy technology, offers superior visualization of vascular details in lesion areas along the airway walls compared to standard white light bronchoscopy. This empowers physicians to detect suspect lesions and characterize their underlying vascular structures for further indications of cancerous activity. Unfortunately, the bronchoscopic video stream suffers from blurring artifacts due to device and patient motions, resulting in low-resolution visualization of lesion areas. To address this problem, we present an image enhancement method for NBI bronchoscopy to improve: 1) visualization of vascular structures; 2) lesion detection; and 3) vessel segmentation. We adapted Real-ESRGAN, a single-image super-resolution network, to enhance bronchoscopic images in real-time. This involved a transfer learning approach to fine-tune a pre-trained model using our public NBI bronchial lesion database. The results, derived from bronchoscopic airway exam videos of 10 lung cancer patients, demonstrate significant improvement in the visual quality of super-resolved frames, particularly in vascular regions. Our quantitative analysis further shows enhanced vessel segmentation and lesion detection accuracy, with increased confidence scores. This method offers a practical, real-time solution for improving the diagnostic utility of NBI bronchoscopy by providing clearer, more detailed images. Thus, we integrated the method into an NBI video analysis system for aiding in the early detection and characterization of bronchial lesions.

Guided bronchoscopy systems, be they image-guided or robotics-assisted, are revolutionizing the field of bronchoscopy, particularly for tasks in peripheral lung regions. To provide guidance, these systems draw on a procedure plan derived from a patient's 3D chest computed tomography (CT) scan. Generally, a procedure plan gives an airway route leading from the trachea to a distal airway close to a diagnostic region of interest (ROI). Unfortunately, the "virtual" chest space, captured by the CT scan used for planning, differs significantly from the "real" chest space encountered during the live procedure. This so-called CT-to-body divergence (CTBD) can produce large guidance errors during a live bronchoscopy, resulting in procedure failure rates on the order of 30%. CTBD arises because the procedure plan relies on a CT scan depicting the chest near total lung capacity (TLC), while the live procedure occurs with the chest near functional residual capacity (FRC), a much lower lung volume. These differences in lung volume lead to mismatches between the guidance system's perceived 3D chest position and bronchoscope's actual chest position. To address this gap, we propose a two-phase method to help mitigate the impact of CTBD on guided bronchoscopy procedures by utilizing spatial information from CT scans obtained at two different lung states. The first phase entails a dual-space procedure planning method that gives two point-by-point synchronized procedure plans: one in TLC space and the other in FRC space. The second phase involves a dual-space guidance procedure, whereby synchronized information in both TLC and FRC spaces are made available during the guided procedure. Results from TLC/FRC CT scan pairs collected for lung cancer patients verify the potential of the method.

Purpose: Early detection of cancer is crucial for lung cancer patients, as it determines disease prognosis. Lung cancer typically starts as bronchial lesions along the airway walls. Recent research has indicated that narrow-band imaging (NBI) bronchoscopy enables more effective bronchial lesion detection than other bronchoscopic modalities. Unfortunately, NBI video can be hard to interpret because physicians currently are forced to perform a time-consuming subjective visual search to detect bronchial lesions in a long airway-exam video. As a result, NBI bronchoscopy is not regularly used in practice. To alleviate this problem, we propose an automatic two-stage real-time method for bronchial lesion detection in NBI video and perform a first-of-its-kind pilot study of the method using NBI airway exam video collected at our institution. Approach: Given a patient's NBI video, the first method stage entails a deep-learning-based object detection network coupled with a multiframe abnormality measure to locate candidate lesions on each video frame. The second method stage then draws upon a Siamese network and a Kalman filter to track candidate lesions over multiple frames to arrive at final lesion decisions. Results: Tests drawing on 23 patient NBI airway exam videos indicate that the method can process an incoming video stream at a real-time frame rate, thereby making the method viable for real-time inspection during a live bronchoscopic airway exam. Furthermore, our studies showed a 93% sensitivity and 86% specificity for lesion detection; this compares favorably to a sensitivity and specificity of 80% and 84% achieved over a series of recent pooled clinical studies using the current time-consuming subjective clinical approach. Conclusion: The method shows potential for robust lesion detection in NBI video at a real-time frame rate. Therefore, it could help enable more common use of NBI bronchoscopy for bronchial lesion detection. (c) 2024 Society of Photo-Optical Instrumentation Engineers (SPIE)

For patients at risk of developing either lung cancer or colorectal cancer, the identification of suspect lesions in endoscopic video is an important procedure. The physician performs an endoscopic exam by navigating an endoscope through the organ of interest, be it the lungs or intestinal tract, and performs a visual inspection of the endoscopic video stream to identify lesions. Unfortunately, this entails a tedious, error-prone search over a lengthy video sequence. We propose a deep learning architecture that enables the real-time detection and segmentation of lesion regions from endoscopic video, with our experiments focused on autofluorescence bronchoscopy (AFB) for the lungs and colonoscopy for the intestinal tract. Our architecture, dubbed ESFPNet, draws on a pretrained Mix Transformer (MiT) encoder and a decoder structure that incorporates a new Efficient Stage-Wise Feature Pyramid (ESFP) to promote accurate lesion segmentation. In comparison to existing deep learning models, the ESFPNet model gave superior lesion segmentation performance for an AFB dataset. It also produced superior segmentation results for three widely used public colonoscopy databases and nearly the best results for two other public colonoscopy databases. In addition, the lightweight ESFPNet architecture requires fewer model parameters and less computation than other competing models, enabling the real-time analysis of input video frames. Overall, these studies point to the combined superior analysis performance and architectural efficiency of the ESFPNet for endoscopic video analysis. Lastly, additional experiments with the public colonoscopy databases demonstrate the learning ability and generalizability of ESFPNet, implying that the model could be effective for region segmentation in other domains.

Because lung cancer is the leading cause of cancer-related deaths globally, early disease detection is vital. To help with this issue, advances in bronchoscopy have brought about three complementary noninvasive video modalities for imaging early-stage bronchial lesions along the airway walls: white-light bronchoscopy (WLB), autofluorescence bronchoscopy (AFB), and narrow-band imaging (NBI). Recent research indicates that performing a multimodal airway exam — i.e., using the three modalities together — potentially enables a more robust disease assessment than any single modality. Unfortunately, to perform a multimodal exam, the physician must manually examine each modality's video stream separately and then mentally correlate lesion observations. This process is not only extremely tedious and skill-dependent, but also poses the risk of missed lesions, thereby reducing diagnostic confidence. What is needed is a methodology and set of tools for easily leveraging the complementary information offered by these modalities. To address this need, we propose a framework for video synchronization and fusion tailored to multimodal bronchoscopic airway examination. Our framework, built into an interactive graphical system, entails a three-step process. First, for each of the three airway exams performed with a given bronchoscopic modality, several key airway video-frame landmarks are noted with respect to the patient's CT-based 3D airway tree model (CT = computed tomography), where the airway tree model serves as a reference space for the entire process. These landmarks create a set of connections between the videos and the airway tree to facilitate subsequent fine registration. Second, the landmark set, along with a set of additional video frames, which either contain detected lesions flagged by two deep-learning-based detection networks or lie between landmarks to help fill surface gaps, are finely registered to the airway tree, using a CT-video-based global registration method. Lastly, the registered frames are mapped and fused, via texture mapping, to the CT-based 3D airway tree's endoluminal surface. This enables sequential revising of synchronized multimodal surface structure and lesion locations through interactive graphical tools along a path navigating the airway tree. Results with patient multimodal bronchoscopic airway exams show the promise of our methods.

The management of lung cancer necessitates robust diagnostic tools, with three-dimensional (3D) computed tomography (CT) imaging and bronchoscopy standing as pivotal complementary resources. Bronchoscopy captures live endobronchial video, providing striking detail of the airway tree's interior, while 3D CT scans contribute extensive anatomical knowledge. A significant gap persists, however, in linking these data-rich sources, such as in the fusion of video data from bronchoscopic airway exams and airway surface data from 3D CT scans. The main issue is the difficulty in simultaneously acquiring depth and camera pose information for bronchoscopic video frames. A solution to this problem can facilitate CT-video fusion/rendering, multimodal registration, and 3D cancer lesion localization. Deep-learning networks have been recently employed to estimate the depth and ego-motion information. Unfortunately, it is challenging to acquire the required training data, consisting of ground-truth pairs of bronchoscopic video frames and corresponding depth maps. Along this line, generative adversarial networks (GANs) have shown promise in domain transformation from CT-based endoluminal surface views into synthesized bronchoscopic frames. These synthesized views are consequently aligned with their CT-derived depth map, generating valuable training data. Nonetheless, such domain transformation techniques fail to utilize frame sequence knowledge and supply no information about the camera's ego-motion. Parallel studies in other domains, such as endoscopy, have emphasized the photometric consistency between adjacent frames to jointly offer depth and ego-motion estimation. Nevertheless, the texture-less and smooth endoluminal surface inside the airway restricts the generation of distinct depth maps with enhanced clarity and detail. To address this problem, we present a self-supervised training strategy that incorporates both domain transformation and photometric consistency for the Monodepth2 deep learning architecture, improving the depth and ego-motion prediction of bronchoscopic video frames. Results drawing on well-registered test data illustrate that the proposed strategy achieves clear and precise prediction. In addition, effective reference scaling factors are summarized from the test dataset, enabling real-world applications, such as 3D surface reconstruction, camera trajectory generation, and fusion between CT and bronchoscopic video.

The examination of suspicious peripheral pulmonary lesions (PPLs) is an important part of lung cancer diagnosis. The physician performs bronchoscopy and then employs radial-probe endobronchial ultrasound (RP-EBUS) to examine and biopsy suspect lesions. Physician skill, however, plays a significant part in the success of these procedures. This has driven the introduction of image-guided bronchoscopy systems. Yet, while such systems enable route planning through the airways to reach a desired region of interest (ROI), the route does not come with the appropriate device maneuvers crucial for instructing the physician on how to use the bronchoscope and the R-EBUS probe during the procedure. Our recently proposed image-guided bronchoscopy system, the Multimodal Virtual Navigation System (MVNS), does offer such instructions for the devices involved. Unfortunately, the system relies on a time-consuming, error-prone, manual approach to generate these device maneuvers. We propose an automatic approach for creating a complete set of device maneuvers for both the bronchoscope and the R-EBUS probe, fully integrating it into the MVNS in the process. Results show that planning the device maneuvers, which previously took on the order of 5 minutes or more per ROI, is reduced to under one second.

Lung cancer tends to be detected at an advanced stage, resulting in a high patient mortality rate. Thus, much recent research has focused on early disease detection Bronchoscopy is the procedure of choice for an effective noninvasive way of detecting early manifestations (bronchial lesions) of lung cancer. In particular, autofluorescence bronchoscopy (AFB) discriminates the autofluorescence properties of normal (green) and diseased tissue (reddish brown) with different colors. Because recent studies show AFB's high sensitivity in searching lesions, it has become a potentially pivotal method in bronchoscopic airway exams. Unfortunately, manual inspection of AFB video is extremely tedious and error prone, while limited effort has been expended toward potentially more robust automatic AFB lesion analysis. We propose a real-time (processing throughput of 27 frames/sec) deep-learning architecture dubbed ESFPNet for accurate segmentation and robust detection of bronchial lesions in AFB video streams. The architecture features an encoder structure that exploits pretrained Mix Transformer (MiT) encoders and an efficient stage-wise feature pyramid (ESFP) decoder structure. Segmentation results from the AFB airway-exam videos of 20 lung cancer patients indicate that our approach gives a mean Dice index = 0.756 and an average Intersection of Union = 0.624, results that are superior to those generated by other recent architectures. Thus, ESFPNet gives the physician a potential tool for confident real-time lesion segmentation and detection during a live bronchoscopic airway exam. Moreover, our model shows promising potential applicability to other domains, as evidenced by its state-of-the-art (SOTA) performance on the CVC-ClinicDB, ETIS-LaribPolypDB datasets, and superior performance on the Kvasir, CVC-ColonDB datasets.

Image-guided bronchoscopy systems and new robotics-assisted bronchoscopy systems are transforming the practice of bronchoscopy. To use such a system, the physician must first create an airway route plan to preselected Regions of Interests (ROIs) using a patient’s chest CT scan, prior to the live procedure. Many unexpected situations arise during the live procedure, however, where the physician must examine a new previously unplanned ROI site — this requires an airway guidance route leading to the new site derived live in real time. We propose a method for deriving an airway route during a live bronchoscopic procedure to any selected site in any imaging view or bronchoscopic video view observed on an assisted-bronchoscopy system’s display. The method includes an interactive graphical tool for managing and selecting new ROI sites and fits within the framework of an assisted-bronchoscopy system. When a site is selected, the methodology draws on the patient’s chest CT scan and a previously derived airway-tree centerline structure to compute the desired airway route in real-time. Subsequently, the physician can then preview the new airway route on the assisted-bronchoscopy system’s display and undertake guided bronchoscopy to the new site. Example results demonstrate the methodology.

Because Radial-Probe Endobronchial Ultrasound (RP-EBUS) can provide real-time confirmation of a suspect peripheral nodule situated outside of the airways, it is widely used during bronchoscopy for lung cancer diagnosis. RP-EBUS, however, tends to be difficult to use effectively, without some form of guidance. Previously, we had prototyped a multimodal image-guided bronchoscopy system that provides guidance during both bronchoscopic navigation and RP-EBUS localization. To use the system, the user first generates a guidance plan offline prior to the live procedure. Later, in the surgical suite, the user then employs the image-guided system to perform the desired multimodal RP-EBUS bronchoscopy, driven by the procedure plan. We now validate this system in a series of live studies. As the first set of end-to-end live system studies, we first tested the system in controlled animal studies. Through these studies, we tested the functionality and feasibility of the system prototype over the standard clinical workflow, without the usual risks associated with live patient procedures. Through these studies, we sharpened the workflow for the prototype and improved user interaction. We then tested the refined system over the standard clinical workflow in our University Hospital’s lung cancer management clinic. This study proved the potential of our system for live clinical usage by demonstrating the safety, feasibility, and functionality of our complete system for guiding RP-EBUS bronchoscopy during peripheral nodule diagnosis.

Narrow-band imaging (NBI) bronchoscopy offers enhanced visualization of microvascular structures in the lung’s epithelium (airway walls). Recent studies suggest that such vessels are helpful in predicting the invasiveness of bronchial lesions. In particular, Shibuya characterized pathological features of lesions and studied their relationship with specific histological stages of lung cancer. We propose a method for identifying these vascular patterns using a small expert-labeled dataset. Our approach is based on a few-shot learning method using a Siamese network to learn and distinguish pathological features of the bronchial vasculature. We achieved better intra-class clustering and inter-class separation in the embedding space compared to a baseline CNN classifier. Further, a 25% increase in the overall accuracy was obtained during testing.

The state-of-the-art procedure for examining the lymph nodes in a lung cancer patient involves using an endobronchial ultrasound (EBUS) bronchoscope. The EBUS bronchoscope integrates two modalities into one device: (1) videobronchoscopy, which gives video images of the airway walls; and (2) convex-probe EBUS, which gives 2D fan-shaped views of extraluminal structures situated outside the airways. During the procedure, the physician first employs videobronchoscopy to navigate the device through the airways. Next, upon reaching a given node’s approximate vicinity, the physician probes the airway walls using EBUS to localize the node. Due to the fact that lymph nodes lie beyond the airways, EBUS is essential for confirming a node’s location. Unfortunately, it is well-documented that EBUS is difficult to use. In addition, while new image-guided bronchoscopy systems provide effective guidance for videobronchoscopic navigation, they offer no assistance for guiding EBUS localization. We propose a method for registering a patient’s chest CT scan to live surgical EBUS views, thereby facilitating accurate image-guided EBUS bronchoscopy. The method entails an optimization process that registers CT-based virtual EBUS views to live EBUS probe views. Results using lung cancer patient data show that the method correctly registered 28/28 (100%) lymph nodes scanned by EBUS, with a mean registration time of 3.4 s. In addition, the mean position and direction errors of registered sites were 2.2 mm and 11.8∘, respectively. In addition, sensitivity studies show the method’s robustness to parameter variations. Lastly, we demonstrate the method’s use in an image-guided system designed for guiding both phases of EBUS bronchoscopy.

During the diagnosis of peripheral pulmonary lesions (PPLs), radial-probe endobronchial ultrasound (RP-EBUS) is often used in combination with bronchoscopy to visualize extraluminal structures and confirm lesion sites. However, due to the steep learning curve, a physician's ability to use RP-EBUS varies greatly in practice. On another front, image-guided bronchoscopy systems are now commonly used to assist with bronchoscopy planning and guidance.(1) A procedure plan is first generated from the patient's CT scan consisting of airway routes leading to each target lesion. Next, during live bronchoscopy, the system helps navigate the bronchoscope along the preplanned airway route close to the target lesion. However, there is no direct modality linkage and guidance between RP-EBUS and the guidance system during image-guided bronchoscopy. We now propose a multimodal image-guided methodology for guiding RP-EBUS consisting of two parts: 1) a CT-based procedure planning method that enables optimal RP-EBUS invocation, localization, preview, and RP-EBUS video simulation. 2) an intra-operative guidance system tailored to RP-EBUS localization of PPLs. We present results demonstrating the proposed system.