Computational tools, such as "digital twin" modeling, are facilitating patient-specific surgical planning for ablative therapies in treating hepatocellular carcinoma. In microwave ablation (MWA), various tissue-specific factors-like tissue perfusion, material properties, and disease state-affect ablative outcomes, which current thermal dosing guidelines often overlook. Moreover, adaptive strategies such as multi-antenna thermal ablation enhance MWA for larger or complex tumors. The biophysical interaction between simultaneously activated multiple ablation probes results in unique, fused ablation zones that differ from individual probe activations at the same power level. Finite element modeling can address this unpredictability but is quite cumbersome to be used in real-time clinically. Advances in machine learning, including PointNet+, offer accurate volumetric predictions as a substitute for finite element modeling, significantly reducing computation time. This study introduces a framework for constructing biophysical digital twin models to forecast multiple ablation volumes, which are then used to train two machine learning models: the Pointnet Vector Model ( PVM) and the Fixed Grid Model (FGM), to enable rapid predictions and demonstrate the clinical viability of a precision medicine approach. Comparison of the PVM and FGM highlights key differences in their predictive accuracy for ablation volumes. The PVM consistently underestimates volumes, with deviations increasing over time: 6.00% at 3 minutes (16.64 cm(3) vs. 17.70 cm(3)), 9.12% at 8 minutes (36.79 cm(3) vs. 40.48 cm(3)), and 13.04% at 15 minutes (49.99 cm(3) vs. 57.50 cm (3)). Histogram analysis reveals an average percentage error near -10%, with outliers up to +/- 20%, alongside variability in spatial representation, especially at the periphery. These issues are more pronounced in non-parallel probe setups. Conversely, the FGM demonstrates better accuracy and stability, with errors under 2% at 3 minutes (17.70 cm(3) vs. 18.00 cm(3)), near-zero at 8 minutes (39.79 cm(3) vs. 40.48 cm(3)), and under 1% at 15 minutes (57.18 cm(3) vs. 57.50 cm(3)). Histogram outliers remain within +/- 8%. Improved spatial alignment in FGM reflects greater robustness, leaving the PVM requiring refinement for complex geometries and reducing predictive biases. Neither model achieves perfect alignment, indicating further methodological advancements are needed. With continued refinement of simulation methodologies and incorporation of more diverse training data, both models have the potential to achieve higher predictive reliability and better address the complexities of ablation volume estimation.

Purpose:Computational methods for image-to-physical registration during surgical guidance frequently rely on sparse point clouds obtained over a limited region of the organ surface. However, soft tissue deformations complicate the ability to accurately infer anatomical alignments from sparse descriptors of the organ surface. The Image-to-Physical Liver Registration Sparse Data Challenge introduced at SPIE Medical Imaging 2019 seeks to characterize the performance of sparse data registration methods on a common dataset to benchmark and identify effective tactics and limitations that will continue to inform the evolution of image-to-physical registration algorithms. Approach:Three rigid and five deformable registration methods were contributed to the challenge. The deformable approaches consisted of two deep learning and three biomechanical boundary condition reconstruction methods. These algorithms were compared on a common dataset of 112 registration scenarios derived from a tissue-mimicking phantom with 159 subsurface validation targets. Target registration errors (TRE) were evaluated under varying conditions of data extent, target location, and measurement noise. Jacobian determinants and strain magnitudes were compared to assess displacement field consistency. Results:Rigid registration algorithms produced significant differences in TRE ranging from 3.8±2.4  mm to 7.7±4.5  mm, depending on the choice of technique. Two biomechanical methods yielded TRE of 3.1±1.8  mm and 3.3±1.9  mm, which outperformed optimal rigid registration of targets. These methods demonstrated good performance under varying degrees of surface data coverage and across all anatomical segments of the liver. Deep learning methods exhibited TRE ranging from 4.3±3.3  mm to 7.6±5.3  mm but are likely to improve with continued development. TRE was weakly correlated among methods, with greatest agreement and field consistency observed among the biomechanical approaches. Conclusions:The choice of registration algorithm significantly impacts registration accuracy and variability of deformation fields. Among current sparse data driven image-to-physical registration algorithms, biomechanical simulations that incorporate task-specific insight into boundary conditions seem to offer best performance.

Emerging computational tools such as healthcare digital twin modeling are enabling the creation of patient-specific surgical planning, including microwave ablation to treat primary and secondary liver cancers. Healthcare digital twins (DTs) are anatomically one-to-one biophysical models constructed from structural, functional, and biomarker-based imaging data to simulate patient-specific therapies and guide clinical decision-making. In microwave ablation (MWA), tissue-specific factors including tissue perfusion, hepatic steatosis, and fibrosis affect therapeutic extent, but current thermal dosing guidelines do not account for these parameters. This study establishes an MR imaging framework to construct three-dimensional biophysical digital twins to predict ablation delivery in livers with 5 levels of fat content in the presence of a tumor. Four microwave antenna placement strategies were considered, and simulated microwave ablations were then performed using 915 MHz and 2450 MHz antennae in Tumor Naive DTs (control), and Tumor Informed DTs at five grades of steatosis. Across the range of fatty liver steatosis grades, fat content was found to significantly increase ablation volumes by approximately 29-l42% in the Tumor Naive and 55-60% in the Tumor Informed DTs in 915 MHz and 2450 MHz antenna simulations. The presence of tumor did not significantly affect ablation volumes within the same steatosis grade in 915 MHz simulations, but did significantly increase ablation volumes within mild-, moderate-, and high-fat steatosis grades in 2450 MHz simulations. An analysis of signed distance to agreement for placement strategies suggests that accounting for patient-specific tumor tissue properties significantly impacts ablation forecasting for the preoperative evaluation of ablation zone coverage.

The ability to accurately account for intraoperative soft tissue deformations has been a longstanding barrier to efficacious translation of image-guided frameworks into abdominal interventions. In surgical applications, few data acquisition systems are amenable to stringent operative workflow constraints, and many are too costly for widespread adoption. Consequently, computational methods for surgical guidance based on sparse data obtained over the organ surface have become prevalent within approaches for image-to-patient alignment of soft tissue. However, the sparse data environment presents an especially challenging algorithmic landscape for accurately inferring deformable anatomical alignments between preoperative and intraoperative organ states from incomplete information sources. This work, presented as a preliminary conclusion to the image-to-physical liver registration sparse data challenge introduced at SPIE Medical Imaging 2019, seeks to characterize the potential for sparse data registration algorithms to achieve high fidelity predictions of intraoperative organ deformations from sparse descriptors of organ surface shape. A total of seven rigid and nonrigid biomechanical and deep learning registration techniques are compared, and the findings suggest that the family of biomechanically simulated boundary condition reconstruction techniques offers a promising opportunity for accurately estimating intraoperative organ deformation states from sparse intraoperative point clouds. Over a common dataset of 112 registration scenarios, this family of deformable registration techniques was found to outperform globally optimal rigid registrations, was robust to varying degrees of surface data coverage, and maintained good performance under added sources of measurement noise. Further analysis investigates error correlations among methods to illuminate sparse data performance within state-of-the-art image-to-physical registration algorithms.

Computational tools, such as "digital twin" modeling, are beginning to enable patient-specific surgical planning of ablative therapies to treat hepatocellular carcinoma. Digital twins models use patient functional data and biomarker imaging to build anatomically accurate models to forecast therapeutic outcomes through simulation, i.e., providing accurate information for guiding clinical decision-making. In microwave ablation (MWA), tissue-specific factors (e.g., tissue perfusion, material properties, disease state, etc.) can affect ablative therapies, but current thermal dosing guidelines do not account for these differences. This study establishes an imaging-data-driven framework to construct digital twin biophysical models to predict ablation extents in livers with varying fat content in MWA. Patient anatomic scans were segmented to develop customized three-dimensional computational biophysical models, and fat-quantification images were acquired to reconstruct spatially accurate biophysical material properties. Simulated patient-specific microwave ablations of homogenous digital-twin models (control) and enhanced digital twin models were performed at four levels of fatty liver disease. When looking at the short diameter (SD), long diameter (LD), ablation volume, and spherical index of the ablation margins - the heterogenous digital-twin models did not produce significantly different ablation margins compared to the control models. Both models produced results that report ablation margins for patients with high-fat livers are larger than low-fat livers (LD of 6.17cm vs. 6.30cm and SD of 2.10 vs. 1.99, respectively). Overall, the results suggest that modeling heterogeneous clinical fatty liver disease using fat-quantitative imaging data has the potential to improve patient specificity for this treatment modality.

Computational tools are beginning to enable patient-specific surgical planning to localize and prescribe thermal dosing for liver cancer ablation therapy. Tissue-specific factors (e.g., tissue perfusion, material properties, disease state, etc.) have been found to affect ablative therapies, but current thermal dosing guidance practices do not account for these differences. Computational modeling of ablation procedures can integrate these sources of patient specificity to guide therapy planning and delivery. This paper establishes an imaging-data-driven framework for patient-specific biophysical modeling to predict ablation extents in livers with varying fat content in the context of microwave ablation (MWA) therapy. Patient anatomic scans were segmented to develop customized three-dimensional computational biophysical models and mDIXON fat-quantification images were acquired and analyzed to establish fat content and determine biophysical properties. Simulated patient-specific microwave ablations of tumor and healthy tissue were performed at four levels of fatty liver disease. Ablation models with greater fat content demonstrated significantly larger treatment volumes compared to livers with less severe disease states. More specifically, the results indicated an eightfold larger difference in necrotic volumes with fatty livers vs. the effects from the presence of more conductive tumor tissue. Additionally, the evolution of necrotic volume formation as a function of the thermal dose was influenced by the presence of a tumor. Fat quantification imaging showed multi-valued spatially heterogeneous distributions of fat deposition, even within their respective disease classifications (e.g., low, mild, moderate, high-fat). Altogether, the results suggest that clinical fatty liver disease levels can affect MWA, and that fat-quantitative imaging data may improve patient specificity for this treatment modality.

Purpose: Intra-Cardiac Echocardiography (ICE) is a powerful imaging modality for guiding cardiac electrophysiology and structural heart interventions. ICE provides real-time observation of anatomy and devices, while enabling direct monitoring of potential complications. In single operator settings, the physician needs to switch back-and-forth between the ICE catheter and therapy device, making continuous ICE support impossible. Two operators setup are therefore sometimes implemented, with the challenge of increase room occupation and cost. Two operator setups are sometimes implemented, but increase procedural costs and room occupation. Methods: ICE catheter robotic control system is developed with automated catheter tip repositioning (i.e. view recovery) method, which can reproduce important views previously navigated to and saved by the user. The performance of the proposed method is demonstrated and evaluated in a combination of heart phantom and animal experiments. Results: Automated ICE view recovery achieved catheter tip position accuracy of 2.09 +/-0.90 mm and catheter image orientation accuracy of 3.93 +/- 2.07 degree in animal studies, and 0.67 +/- 0.79 mm and 0.37 +/- 0.19 degree in heart phantom studies, respectively. Our proposed method is also successfully used during transeptal puncture in animals without complications, showing the possibility for fluoro-less transeptal puncture with ICE catheter robot. Conclusion: Robotic ICE imaging has the potential to provide precise and reproducible anatomical views, which can reduce overall execution time, labor burden of procedures, and x-ray usage for a range of cardiac procedures. Keywords: Automated View Recovery, Path Planning, Intra-cardiac echocardiography (ICE), Catheter, Tendon-driven manipulator, Cardiac Imaging

Tendon-sheath-driven manipulators (TSM) are widely used in minimally invasive surgical systems due to their long, thin shape, flexibility, and compliance making them easily steerable in narrow or tortuous environments. Many commercial TSM-based medical devices have non-linear phenomena resulting from their composition such as backlash hysteresis and dead zone, which lead to a considerable challenge for achieving precise control of the end effector pose. However, many recent works in the literature do not consider the combined effects and compensation of these phenomena, and less focus on practical ways to identify model parameters in realistic conditions. This paper proposes a simplified piecewise linear model to construct both backlash hysteresis and dead zone compensators together. Further, a practical method is introduced to identify model parameters using motor current from a robotic controller for the TSM. Our proposed methods are validated with multiple Intra-cardiac Echocardiography (ICE) catheters, which are typical commercial example of TSM, by periodic and non-periodic motions. Our results show that the errors from backlash hysteresis and dead zone are considerably reduced and therefore the accuracy of robotic control is improved when applying the presented methods.

In this work, a preliminary effort toward a novel multi-physics modeling framework is presented that combines computational approaches in soft-tissue biomechanics, and bioelectric/bioheat transport to create a patient-specific, image-data driven guidance platform to improve localization and predict thermal dose extent for microwave ablation. More specifically, a finite element modeling and optimization approach for microwave ablation delivery is driven by sparse intra-procedural geometric digitization, and pre-procedural imaging data for providing image-to-physical registration, and dielectric property estimates, respectively. In a series of mock liver phantom experiments, the framework is explored herein. Results indicate superior localization using our non-rigid registration approach, and accurate prediction of lesion formation using our image-data-driven approach to ablation forecasting. Results also provide insight on the impact of localization and material property inaccuracies with respect to therapy delivery and show systematic and considerable degradation of lesion-to-target overlap.

Intra-cardiac Echocardiography (ICE) is a powerful imaging modality for guiding electrophysiology and structural heart interventions. ICE provides real-time observation of anatomy, catheters, and emergent complications. However, this increased reliance on intraprocedural imaging creates a high cognitive demand on physicians who can often serve as interventionalist and imager. We present a robotic manipulator for ICE catheters to assist physicians with imaging and serve as a platform for developing processes for procedural automation. Herein, we introduce two application modules towards these goals: (1) a view recovery process that allows physicians to save views during intervention and automatically return with the push of a button and (2) a data-driven approach to compensate kinematic model errors that result from non-linear behaviors in catheter bending, providing more precise control of the catheter tip. View recovery is validated by repeated catheter positioning in cardiac phantom and animal experiments with position- and image-based analysis. We present a simplified calibration approach for error compensation and verify with complex rotation of the catheter in benchtop and phantom experiments under varying realistic curvature conditions. Results support that a robotic manipulator for ICE can provide an efficient and reproducible tool, potentially reducing execution time and promoting greater utilization of ICE imaging.

We compare a surface-driven, model-based deformation correction method to a clinically relevant rigid registration approach within the application of image-guided microwave ablation for the purpose of demonstrating improved localization and antenna placement in a deformable hepatic phantom. Furthermore, we present preliminary computational modeling of microwave ablation integrated within the navigational environment to lay the groundwork for a more comprehensive procedural planning and guidance framework. To achieve this, we employ a simple, retrospective model of microwave ablation after registration, which allows a preliminary evaluation of the combined therapeutic and navigational framework. When driving registrations with full organ surface data (i.e., as could be available in a percutaneous procedure suite), the deformation correction method improved average ablation antenna registration error by 58.9% compared to rigid registration (i.e., 2.5 +/- 1.1 mm, 5.6 +/- 2.3 mm of average target error for corrected and rigid registration, respectively) and on average improved volumetric overlap between the modeled and ground-truth ablation zones from 67.0 +/- 11.8% to 85.6 +/- 5.0% for rigid and corrected, respectively. Furthermore, when using sparse-surface data (i.e., as is available in an open surgical procedure), the deformation correction improved registration error by 38.3% and volumetric overlap from 64.8 +/- 12.4% to 77.1 +/- 8.0% for rigid and corrected, respectively. We demonstrate, in an initial phantom experiment, enhanced navigation in image-guided hepatic ablation procedures and identify a clear multiphysics pathway toward a more comprehensive thermal dose planning and deformation-corrected guidance framework. (C) 2019 Society of Photo-Optical Instrumentation Engineers (SPIE)

Over the last 25 years, the number of papers written that involve image guidance, liver, and registration has, on average, doubled every 6-8 years. While image guidance has a long history within the neurosurgical theatre, it’s translation to other soft-tissue organs such as the liver has been slower given the inherent difficulty in image-to-physical registration. More specifically, deformations have been recognized to compromise image guidance fidelity in virtually all soft-tissue image guidance applications. As a result, an active area of investigation is the development of sparse-data-driven nonrigid image-to-physical liver registration techniques to compensate for deformation and provide accurate localization for image guided liver surgery. In this work we have leveraged our extensive human-to-phantom registration testing framework based on the work in [1] and Amazon Web Services to create a sparse data challenge for the image guided liver surgery community (https://sparsedatachallenge.org/). Our sparse data challenge will allow research groups from across the world to extensively test their approaches on common data and have quantitative accuracy measurements provided for assessment of fidelity. Welcome to the Sparse Data Challenge for image-to-physical liver registration assessment.

OBJECTIVE:Accurate prospective modeling of microwave ablation (MWA) procedures can provide powerful planning and navigational information to physicians. However, patient-specific tissue properties are generally unavailable and can vary based on factors such as relative perfusion and state of disease. Therefore, a need exists for modeling frameworks that account for variations in tissue properties. METHODS:In this study, we establish an inverse modeling approach to reconstruct a set of tissue properties that best fit the model-predicted and observed ablation zone extents in a series of phantoms of varying fat content. We then create a model of these tissue properties as a function of fat content and perform a comprehensive leave-one-out evaluation of the predictive property model. Furthermore, we validate the inverse-model predictions in a separate series of phantoms that include co-recorded temperature data. RESULTS:This model-based approach yielded thermal profiles in close agreement with experimental measurements in the series of validation phantoms (average root-mean-square error of 4.8 °C). The model-predicted ablation zones showed compelling overlap with observed ablations in both the series of validation phantoms (93.4 ± 2.2%) and the leave-one-out cross validation study (86.6 ± 5.3%). These results demonstrate an average improvement of 17.3% in predicted ablation zone overlap when comparing the presented property-model to properties derived from phantom component volume fractions. CONCLUSION:These results demonstrate accurate model-predicted ablation estimates based on image-driven determination of tissue properties. SIGNIFICANCE:The work demonstrates, as a proof-of-concept, that physical modeling parameters can be linked with quantitative medical imaging to improve the utility of predictive procedural modeling for MWA.

Soft tissue deformation can be a major source of error for image-guided interventions. Deformations associated with laparoscopic liver surgery can be substantially different from those concomitant with open approaches due to intraoperative practices such as abdominal insufflation and variable degrees of mobilization from the supporting ligaments of the liver. This technical note outlines recent contributions towards nonrigid registration for laparoscopic liver surgery published in the Journal of Medical Imaging special issue on image-guided procedures, robotic interventions, and modeling [10]. In particular, we review (1) a characterization of intraoperative liver deformation from clinically-acquired sparse digitizations of the organ surface through a series of laparoscopic-to-open conversions, and (2) a novel deformation correction strategy that leverages a set of control points placed across anatomical regions of mechanical support provided to the organ. Perturbations of these control points on a finite element model were used to iteratively reconstruct the intraoperative deformed organ shape from sparse measurements of the liver surface. These characterization and correction methods for laparoscopic deformation were applied to a retrospective clinical series of 25 laparoscopic-to-open conversions performed under image guidance and a phantom validation framework.

In open image-guided liver surgery (IGLS), a sparse representation of the intraoperative organ surface can be acquired to drive image-to-physical registration. We hypothesize that uncharacterized error induced by variation in the collection patterns of organ surface data limits the accuracy and robustness of an IGLS registration. Clinical validation of such registration methods is challenged due to the difficulty in obtaining data representative of the true state of organ deformation. We propose a novel human-to-phantom validation framework that transforms surface collection patterns from in vivo IGLS procedures (n = 13) onto a well-characterized hepatic deformation phantom for the purpose of validating surface-driven, volumetric nonrigid registration methods. An important feature of the approach is that it centers on combining workflow-realistic data acquisition and surgical deformations that are appropriate in behavior and magnitude. Using the approach, we investigate volumetric target registration error (TRE) with both current rigid IGLS and our improved nonrigid registration methods. Additionally, we introduce a spatial data resampling approach to mitigate the workflow-sensitive sampling problem. Using our human-to-phantom approach, TRE after routine rigid registration was 10.9 ± 0.6 mm with a signed closest point distance associated with residual surface fit in the range of ±10 mm, highly representative of open liver resections. After applying our novel resampling strategy and improved deformation correction method, TRE was reduced by 51%, i.e., a TRE of 5.3 ± 0.5 mm. This paper reported herein realizes a novel tractable approach for the validation of image-to-physical registration methods and demonstrates promising results for our correction method.

In image-guided liver surgery (IGLS), sparse representations of the anterior organ surface may be collected intraoperatively to drive image-to-physical space registration. Soft tissue deformation represents a significant source of error for IGLS techniques. This work investigates the impact of surface data quality on current surface based IGLS registration methods. In this work, we characterize the robustness of our IGLS registration methods to noise in organ surface digitization. We study this within a novel human-to-phantom data framework that allows a rapid evaluation of clinically realistic data and noise patterns on a fully characterized hepatic deformation phantom. Additionally, we implement a surface data resampling strategy that is designed to decrease the impact of differences in surface acquisition. For this analysis, n=5 cases of clinical intraoperative data consisting of organ surface and salient feature digitizations from open liver resection were collected and analyzed within our human-to-phantom validation framework. As expected, results indicate that increasing levels of noise in surface acquisition cause registration fidelity to deteriorate. With respect to rigid registration using the raw and resampled data at clinically realistic levels of noise (i.e. a magnitude of 1.5 mm), resampling improved TRE by 21%. In terms of nonrigid registration, registrations using resampled data outperformed the raw data result by 14% at clinically realistic levels and were less susceptible to noise across the range of noise investigated. These results demonstrate the types of analyses our novel human-to-phantom validation framework can provide and indicate the considerable benefits of resampling strategies.

Background. Although systems of 3-dimensional image-guided surgery are a valuable adjunct across numerous procedures, differences in organ shape between that reflected in the preoperative image data and the intraoperative state can compromise the fidelity of such guidance based on the image. In this work, we assessed in real time a novel, 3-dimensional image-guided operation platform that incorporates soft tissue deformation.Methods. A series of 125 alignment evaluations were performed across 20 patients. During the operation, the surgeon assessed the liver by swabbing an optically tracked stylus over the liver surface and viewing the image-guided operation display. Each patient had approximately 6 intraoperative comparative evaluations. For each assessment, 1 of only 2 types of alignments were considered: conventional rigid and novel deformable. The series of alignment types used was randomized and blinded to the surgeon. The surgeon provided a rating, R, from -3 to +3 for each display compared with the previous display, whereby a negative rating indicated degradation in fidelity and a positive rating an improvement.Results. A statistical analysis of the series of rating data by the clinician indicated that the surgeons were able to perceive an improvement (defined as a R > I) of the model-based registration over the rigid registration (P = .01) as well as a degradation (defined as R < -1) when the rigid registration was compared with the novel deformable guidance information (P = .03).Conclusion. This study provides evidence of the benefit of deformation correction in providing an accurate location for the liver for use in image-guided surgery systems.

Abstract. Laparoscopic liver surgery is challenging to perform due to a compromised ability of the surgeon to localize subsurface anatomy in the constrained environment. While image guidance has the potential to address this barrier, intraoperative factors, such as insufflation and variable degrees of organ mobilization from supporting ligaments, may generate substantial deformation. The severity of laparoscopic deformation in humans has not been characterized, and current laparoscopic correction methods do not account for the mechanics of how intraoperative deformation is applied to the liver. We first measure the degree of laparoscopic deformation at two insufflation pressures over the course of laparoscopic-to-open conversion in 25 patients. With this clinical data alongside a mock laparoscopic phantom setup, we report a biomechanical correction approach that leverages anatomically load-bearing support surfaces from ligament attachments to iteratively reconstruct and account for intraoperative deformations. Laparoscopic deformations were significantly larger than deformations associated with open surgery, and our correction approach yielded subsurface target error of 6.7±1.3  mm and surface error of 0.8±0.4  mm using only sparse surface data with realistic surgical extent. Laparoscopic surface data extents were examined and found to impact registration accuracy. Finally, we demonstrate viability of the correction method with clinical data.