Computational modeling and simulation (CM&S) is a powerful tool that can be used to support the development, evaluation, and regulatory authorization of medical devices. CM&S can provide valuable insights into device performance, safety, and effectiveness, as well as reduce the need for animal or human testing. Computational models are, however, idealized digital representations that often have many assumptions and need to be credible before they are used in decision making that could incur patient harm. While the medical device community has made great strides to advance the use of CM&S, a number of challenges remain. To begin addressing these challenges, the US Food and Drug Administration (FDA) and the Medical Device Innovation Consortium (MDIC) co-sponsored theFDA/MDIC Symposium on Computational Modeling and Simulationon April 16-17, 2024 in College Park, Maryland, USA, where attendees from around the world convened to hear from leaders in the field through a unique blend of invited presentations and interactive panel discussions. The symposium agenda covered several major themes, including credibility considerations for CM&S used across the medical device total product life cycle, practical examples of performing model credibility assessment, and the use of CM&S for clinical decision making and the emerging areas ofin silicoclinical trials and digital twins. The objective of this article is to summarize the major takeaways of the symposium. We first provide an overview of the invited presentations followed by summaries of the topics covered during the interactive panel discussions. In doing so, we highlight the main takeaways and identify areas in which panelists had shared perspectives or differences of opinion. Next, we present the results of a survey conducted at the symposium that sought attendees' perspectives on different aspects of medical device CM&S. Finally, we conclude by summarizing the major outcomes of the symposium, including areas where more work and investment are needed to advance the field.

Branch pulmonary artery (BPA) reconstruction is associated with high reintervention rates. We present in vitro validation of a patch-planning workflow accounting for vessel prestretch, tissue properties, and suture uptake, to achieve targeted reconstructed dimensions. Two physiologically compliant, centrally stenosed BPA silicone models were created to represent neonatal and child-age repairs. Preoperative CTs were segmented to create 3D models for virtual planning. Patches were designed to restore stenotic regions to target diameters under two physiological pressure extremes, accounting for model and patch material distensibility. Finite-element simulation determined the loaded flat patch configuration. Analytical transformation produced unloaded designs for patches planned at high pressure. Designs were laser projected onto patch surfaces, patches sutured, and postoperative CTs acquired (n ≥ 3 per model/pressure/material). Virtual model credibility was evaluated following a recent credibility assessment framework. Patches designed for low vs. high pressure differed in size by up to 25

Computational models of patients and medical devices can be combined to perform an in silico clinical trial (ISCT) to investigate questions related to device safety and/or effectiveness across the total product life cycle. ISCTs can potentially accelerate product development by more quickly informing device design and testing or they could be used to refine, reduce, or in some cases to completely replace human subjects in a clinical trial. There are numerous potential benefits of ISCTs. An important caveat, however, is that an ISCT is a virtual representation of the real world that has to be shown to be credible before being relied upon to make decisions that have the potential to cause patient harm. There are many challenges to establishing ISCT credibility. ISCTs can integrate many different submodels that potentially use different modeling types (e.g., physics-based, data-driven, rule-based) that necessitate different strategies and approaches for generating credibility evidence. ISCT submodels can include those for the medical device, the patient, the interaction of the device and patient, generating virtual patients, clinical decision making and simulating an intervention (e.g., device implantation), and translating acute physics-based simulation outputs to health-related clinical outcomes (e.g., device safety and/or effectiveness endpoints). Establishing the credibility of each ISCT submodel is challenging, but is nonetheless important because inaccurate output from a single submodel could potentially compromise the credibility of the entire ISCT. The objective of this study is to begin addressing some of these challenges and to identify general strategies for establishing ISCT credibility. Most notably, we propose a hierarchical approach for assessing the credibility of an ISCT that involves systematically gathering credibility evidence for each ISCT submodel in isolation before demonstrating credibility of the full ISCT. Also, following FDA Guidance for assessing computational model credibility, we provide suggestions for ways to clearly describe each of the ISCT submodels and the full ISCT, discuss considerations for performing an ISCT model risk assessment, identify common challenges to demonstrating ISCT credibility, and present strategies for addressing these challenges using our proposed hierarchical approach. Finally, in the Appendix we illustrate the many concepts described here using a hypothetical ISCT example.

In silico clinical trials (ISCTs) are an emerging method in modeling and simulation where medical interventions are evaluated using computational models of patients. ISCTs have the potential to provide cost-effective, time-efficient, and ethically favorable alternatives for evaluating the safety and effectiveness of medical devices. However, ensuring the credibility of ISCT results is a significant challenge. This paper aims to identify unique considerations for assessing the credibility of ISCTs and proposes an ISCT credibility assessment workflow based on recently published model assessment frameworks. First, we review various ISCTs described in the literature, carefully selected to showcase the range of methodological options available. These studies cover a wide variety of devices, reasons for conducting ISCTs, patient model generation approaches including subject-specific versus ‘synthetic’ virtual patients, complexity levels of devices and patient models, incorporation of clinician or clinical outcome models, and methods for integrating ISCT results with real-world clinical trials. We next discuss how verification, validation, and uncertainty quantification apply to ISCTs, considering the range of ISCT approaches identified. Based on our analysis, we then present a hierarchical workflow for assessing ISCT credibility, using a general credibility assessment framework recently published by the FDA’s Center for Devices and Radiological Health. Overall, this work aims to promote standardization in ISCTs and contribute to the wider adoption and acceptance of ISCTs as a reliable tool for evaluating medical devices.

Mathematical models have been an important tool during the COVID-19 pandemic, for example to predict demand of critical resources such as medical devices, personal protective equipment and diagnostic tests. Many COVID-19 models have been developed. However, there is relatively little information available regarding reliability of model predictions. Here we present a general model validation framework for epidemiological models focused around predictive capability for questions relevant to decision-making end-users. COVID-19 models are typically comprised of multiple releases, and provide predictions for multiple localities, and these characteristics are systematically accounted for in the framework, which is based around a set of validation scores or metrics that quantify model accuracy of specific quantities of interest including: date of peak, magnitude of peak, rate of recovery, and monthly cumulative counts. We applied the framework to retrospectively assess accuracy of death predictions for four COVID-19 models, and accuracy of hospitalization predictions for one COVID-19 model (models for which sufficient data was publicly available). When predicting date of peak deaths, the most accurate model had errors of approximately 15 days or less, for releases 3-6 weeks in advance of the peak. Death peak magnitude relative errors were generally in the 50% range 3-6 weeks before peak. Hospitalization predictions were less accurate than death predictions. All models were highly variable in predictive accuracy across regions. Overall, our framework provides a wealth of information on the predictive accuracy of epidemiological models and could be used in future epidemics to evaluate new models or support existing modeling methodologies, and thereby aid in informed model-based public health decision making. The code for the validation framework is available at https://doi.org/10.5281/zenodo.7102854.

There are a variety of difficulties in evaluating clinical cardiac mapping systems, most notably the inability to record the transmembrane potential throughout the entire heart during patient procedures which prevents the comparison to a relevant "gold standard". Cardiac mapping systems are comprised of hardware and software elements including sophisticated mathematical algorithms, both of which continue to undergo rapid innovation. The purpose of this study is to develop a computational modeling framework to evaluate the performance of cardiac mapping systems. The framework enables rigorous evaluation of a mapping system's ability to localize and characterize (i.e., focal or reentrant) arrhythmogenic sources in the heart. The main component of our tool is a library of computer simulations of various dynamic patterns throughout the entire heart in which the type and location of the arrhythmogenic sources are known. Our framework allows for performance evaluation for various electrode configurations, heart geometries, arrhythmias, and electrogram noise levels and involves blind comparison of mapping systems against a "silver standard" comprised of computer simulations in which the precise transmembrane potential patterns throughout the heart are known. A feasibility study was performed using simulations of patterns in the human left atria and three hypothetical virtual catheter electrode arrays. Activation times (AcT) and patterns (AcP) were computed for three virtual electrode arrays: two basket arrays with good and poor contact and one high-resolution grid with uniform spacing. The average root mean squared difference of AcTs of electrograms and those of the nearest endocardial action potential was less than 1 ms and therefore appears to be a poor performance metric. In an effort to standardize performance evaluation of mapping systems a novel performance metric is introduced based on the number of AcPs identified correctly and those considered spurious as well as misclassifications of arrhythmia type; spatial and temporal localization accuracy of correctly identified patterns was also quantified. This approach provides a rigorous quantitative analysis of cardiac mapping system performance. Proof of concept of this computational evaluation framework suggests that it could help safeguard that mapping systems perform as expected as well as provide estimates of system accuracy.

As decisions in drug development increasingly rely on predictions from mechanistic systems models, assessing the predictive capability of such models is becoming more important. Several frameworks for the development of quantitative systems pharmacology (QSP) models have been proposed. In this paper, we add to this body of work with a framework that focuses on the appropriate use of qualitative and quantitative model evaluation methods. We provide details and references for those wishing to apply these methods, which include sensitivity and identifiability analyses, as well as concepts such as validation and uncertainty quantification. Many of these methods have been used successfully in other fields, but are not as common in QSP modeling. We illustrate how to apply these methods to evaluate QSP models, and propose methods to use in two case studies. We also share examples of misleading results when inappropriate analyses are used.

Survival rates for sudden cardiac death treated with external defibrillation are estimated to be up to five times greater compared to cardio-pulmonary resuscitation alone. Computational modeling can be used to investigate the relationship between patch location and defibrillation efficacy. However, credibility of model predictions is unclear. The aims of this paper are to (1) assess credibility of a commonly used computational approach for predicting impact of patch relocation on defibrillation efficacy; and (2) provide a concrete biomedical example of a model validation study with supporting applicability analysis, to systematically assess the relevance of the validation study for a proposed model context of use (COU). An electrostatic heart and torso computational model was developed. Simulations were compared against experimental recordings from a swine subject with external patches and multiple body surface and intracardiac recording electrodes. The applicability of this swine validation study to the human COU was assessed using an applicability analysis framework. Knowledge gaps identified by the applicability analysis were addressed using sensitivity analyses. In the swine validation study, quantitative agreement (R2 = 0.85) was observed between predicted and observed potentials at both surface and intracardiac electrodes using a left-right patch placement. Applicability analysis identified uncertainty in tissue conductivities as one of the main potential sources of unreliability; however, a sensitivity the analysis demonstrated that uncertainty in conductivity parameters had relatively little impact on model predictions (less than 10% relative change for twofold conductivity changes). We believe the results support pursuing human simulations further to evaluate impact of patch relocation.

There is an inherent tension in Quantitative Systems Pharmacology (QSP) between the need to incorporate mathematical descriptions of complex physiology and drug targets with the necessity of developing robust, predictive and well-constrained models. In addition to this, there is no “gold standard” for model development and assessment in QSP. Moreover, there can be confusion over terminology such as model and parameter identifiability; complex and simple models; virtual populations; and other concepts, which leads to potential miscommunication and misapplication of methodologies within modeling communities, both the QSP community and related disciplines. This perspective article highlights the pros and cons of using simple (often identifiable) vs. complex (more physiologically detailed but often non-identifiable) models, as well as aspects of parameter identifiability, sensitivity and inference methodologies for model development and analysis. The paper distills the central themes of the issue of identifiability and optimal model size and discusses open challenges.

BACKGROUND:Left ventricular assist devices (LVADs) are implantable pumps that act as a life support therapy for patients with severe heart failure. Despite improving the survival rate, LVAD therapy can carry major complications. Particularly, the flow distortion introduced by the LVAD in the left ventricle (LV) may induce thrombus formation. While previous works have used numerical models to study the impact of multiple variables in the intra-LV stagnation regions, a comprehensive validation analysis has never been executed. The main goal of this work is to present a model of the LV-LVAD system and to design and follow a verification, validation and uncertainty quantification (VVUQ) plan based on the ASME V&V40 and V&V20 standards to ensure credible predictions. METHODS:The experiment used to validate the simulation is the SDSU cardiac simulator, a bench mock-up of the cardiovascular system that allows mimicking multiple operation conditions for the heart-LVAD system. The numerical model is based on Alya, the BSC's in-house platform for numerical modelling. Alya solves the Navier-Stokes equation with an Arbitrary Lagrangian-Eulerian (ALE) formulation in a deformable ventricle and includes pressure-driven valves, a 0D Windkessel model for the arterial output and a LVAD boundary condition modeled through a dynamic pressure-flow performance curve. The designed VVUQ plan involves: (a) a risk analysis and the associated credibility goals; (b) a verification stage to ensure correctness in the numerical solution procedure; (c) a sensitivity analysis to quantify the impact of the inputs on the four quantities of interest (QoIs) (average aortic root flow [Formula: see text], maximum aortic root flow [Formula: see text], average LVAD flow [Formula: see text], and maximum LVAD flow [Formula: see text]); (d) an uncertainty quantification using six validation experiments that include extreme operating conditions. RESULTS:Numerical code verification tests ensured correctness of the solution procedure and numerical calculation verification showed a grid convergence index (GCI)95% <3.3%. The total Sobol indices obtained during the sensitivity analysis demonstrated that the ejection fraction, the heart rate, and the pump performance curve coefficients are the most impactful inputs for the analysed QoIs. The Minkowski norm is used as validation metric for the uncertainty quantification. It shows that the midpoint cases have more accurate results when compared to the extreme cases. The total computational cost of the simulations was above 100 [core-years] executed in around three weeks time span in Marenostrum IV supercomputer. CONCLUSIONS:This work details a novel numerical model for the LV-LVAD system, that is supported by the design and execution of a VVUQ plan created following recognised international standards. We present a methodology demonstrating that stringent VVUQ according to ASME standards is feasible but computationally expensive.

AbstractPhysiological closed-loop controlled (PCLC) medical devices monitor and automatically adjust the patient’s condition by using physiological variables as feedback, ideally with minimal human intervention to achieve the target levels set by a clinician. PCLC devices present a challenge when it comes to evaluating their performance, where conducting large clinical trials can be expensive. Virtual physiological patients simulated by validated mathematical models can be utilized to obtain pre-clinical evidence of safety and assess the performance of the PCLC medical device during normal and worst-case conditions that are unlikely to happen in a limited clinical trial. A physiological variable that plays a major role during fluid resuscitation is heart rate (HR). For in silico assessment of PCLC medical devices regarding fluid perturbation, there is currently no mathematical model of HR validated in terms of its predictive capability performance. This paper develops and validates a mathematical model of HR response using data collected from sheep subjects undergoing hemorrhage and fluid infusion. The model proved to be accurate in estimating the HR response to fluid perturbation, where averaged between 21 calibration datasets, the fitting performance showed a normalized root mean square error (NRMSE) of $$7.41 \pm 2.8 \%$$ 7.41 ± 2.8 % . The model was also evaluated in terms of model predictive capability performance via a leave-one-out procedure (21 subjects) and an independent validation dataset (6 subjects). Two different virtual cohort generation tools were used in each validation analysis. The generated envelope of virtual subjects robustly met the defined acceptance criteria, in which $$95\%$$ 95 % of the testing datasets presented simulated HR patterns that were within a deviation of 50% from the observed data. In addition, out of 16000 and 18522 simulated subjects for the leave-one-out and independent datasets, the model was able to generate at least one virtual subject that was close to the real subject within an error margin of $$9.56 \pm 3.15\%$$ 9.56 ± 3.15 % and $$11.1 \pm 1.22\%$$ 11.1 ± 1.22 % NRMSE, respectively. In conclusion, the model can generate valid virtual HR physiological responses to fluid perturbation and be incorporated into future non-clinical simulated testing setups for assessing PCLC devices intended for fluid resuscitation.

Reliable and robust simulation of individual patients using patient-specific models (PSMs) is one of the next frontiers for modeling and simulation (M&S) in healthcare. PSMs, which form the basis of digital twins, can be employed as clinical tools to, for example, assess disease state, predict response to therapy, or optimize therapy. They may also be used to construct virtual cohorts of patients, for in silico evaluation of medical product safety and/or performance. Methods and frameworks have recently been proposed for evaluating the credibility of M&S in healthcare applications. However, such efforts have generally been motivated by models of medical devices or generic patient models; how best to evaluate the credibility of PSMs has largely been unexplored. The aim of this paper is to understand and demonstrate the credibility assessment process for PSMs using patient-specific cardiac electrophysiological (EP) modeling as an exemplar. We first review approaches used to generate cardiac PSMs and consider how verification, validation, and uncertainty quantification (VVUQ) apply to cardiac PSMs. Next, we execute two simulation studies using a publicly available virtual cohort of 24 patient-specific ventricular models, the first a multi-patient verification study, the second investigating the impact of uncertainty in personalized and non-personalized inputs in a virtual cohort. We then use the findings from our analyses to identify how important characteristics of PSMs can be considered when assessing credibility with the approach of the ASME V&V40 Standard, accounting for PSM concepts such as inter- and intra-user variability, multi-patient and “every-patient” error estimation, uncertainty quantification in personalized vs non-personalized inputs, clinical validation, and others. The results of this paper will be useful to developers of cardiac and other medical image based PSMs, when assessing PSM credibility.

Subject-specific mathematical models for prediction of physiological parameters such as blood volume, cardiac output, and blood pressure in response to hemorrhage have been developed. In silico studies using these models may provide an effective tool to generate pre-clinical safety evidence for medical devices and help reduce the size and scope of animal studies that are performed prior to initiation of human trials. To achieve such a goal, the credibility of the mathematical model must be established for the purpose of pre-clinical in silico testing. In this work, the credibility of a subject-specific mathematical model of blood volume kinetics intended to predict blood volume response to hemorrhage and fluid resuscitation during fluid therapy was evaluated. A workflow was used in which: (i) the foundational properties of the mathematical model such as structural identifiability were evaluated; (ii) practical identifiability was evaluated both pre- and post-calibration, with the pre-calibration results used to determine an optimal splitting of experimental data into calibration and validation datasets; (iii) uncertainty in model parameters and the experimental uncertainty were quantified for each subject; and (iv) the uncertainty was propagated through the blood volume kinetics model and its predictive capability was evaluated via validation tests. The mathematical model was found to be structurally identifiable. Pre-calibration identifiability analysis led to splitting the 180 min of time series data per subject into 50 and 130 min calibration and validation windows, respectively. The average root mean squared error of the mathematical model was 12.6% using the calibration window of (0 min, 50 min). Practical identifiability was established post-calibration after fixing one of the parameters to a nominal value. In the validation tests, 82 and 75% of the subject-specific mathematical models were able to correctly predict blood volume response when predictive capability was evaluated at 180 min and at the time when amount of infused fluid equals fluid loss.

Providing therapies tailored to each patient is the vision of precision medicine, enabled by the increasing ability to capture extensive data about individual patients. In this position paper, we argue that the second enabling pillar towards this vision is the increasing power of computers and algorithms to learn, reason, and build the 'digital twin' of a patient. Computational models are boosting the capacity to draw diagnosis and prognosis, and future treatments will be tailored not only to current health status and data, but also to an accurate projection of the pathways to restore health by model predictions. The early steps of the digital twin in the area of cardiovascular medicine are reviewed in this article, together with a discussion of the challenges and opportunities ahead. We emphasize the synergies between mechanistic and statistical models in accelerating cardiovascular research and enabling the vision of precision medicine.

Uncertainty quantification (UQ) is a vital step in using mathematical models and simulations to take decisions. The field of cardiac simulation has begun to explore and adopt UQ methods to characterize uncertainty in model inputs and how that propagates through to outputs or predictions; examples of this can be seen in the papers of this issue. In this review and perspective piece, we draw attention to an important and under-addressed source of uncertainty in our predictions-that of uncertainty in the model structure or the equations themselves. The difference between imperfect models and reality is termed model discrepancy, and we are often uncertain as to the size and consequences of this discrepancy. Here, we provide two examples of the consequences of discrepancy when calibrating models at the ion channel and action potential scales. Furthermore, we attempt to account for this discrepancy when calibrating and validating an ion channel model using different methods, based on modelling the discrepancy using Gaussian processes and autoregressive-moving-average models, then highlight the advantages and shortcomings of each approach. Finally, suggestions and lines of enquiry for future work are provided. This article is part of the theme issue 'Uncertainty quantification in cardiac and cardiovascular modelling and simulation'.

Chaste (Cancer, Heart And Soft Tissue Environment) is an open source simulation package for the numerical solution of mathematical models arising in physiology and biology. To date, Chaste development has been driven primarily by applications that include continuum modelling of cardiac electrophysiology (‘Cardiac Chaste’), discrete cell-based modelling of soft tissues (‘Cell-based Chaste’), and modelling of ventilation in lungs (‘Lung Chaste’). Cardiac Chaste addresses the need for a high-performance, generic, and verified simulation framework for cardiac electrophysiology that is freely available to the scientific community. Cardiac chaste provides a software package capable of realistic heart simulations that is efficient, rigorously tested, and runs on HPC platforms. Cell-based Chaste addresses the need for efficient and verified implementations of cell-based modelling frameworks, providing a set of extensible tools for simulating biological tissues. Computational modelling, along with live imaging techniques, plays an important role in understanding the processes of tissue growth and repair. A wide range of cell-based modelling frameworks have been developed that have each been successfully applied in a range of biological applications. Cell-based Chaste includes implementations of the cellular automaton model, the cellular Potts model, cell-centre models with cell representations as overlapping spheres or Voronoi tessellations, and the vertex model. Lung Chaste addresses the need for a novel, generic and efficient lung modelling software package that is both tested and verified. It aims to couple biophysically-detailed models of airway mechanics with organ-scale ventilation models in a package that is freely available to the scientific community. Chaste is designed to be modular and extensible, providing libraries for common scientific computing infrastructure such as linear algebra operations, finite element meshes, and ordinary and partial differential equation solvers. This infrastructure is used by libraries for specific applications, such as continuum mechanics, cardiac models, and cell-based models. The software engineering techniques used to develop Chaste are intended to ensure code quality, re-usability and reliability. Primary applications of the software include cardiac and respiratory physiology, cancer and developmental biology

Patient-specific cardiac models are now being used to guide therapies. The increased use of patient-specific cardiac simulations in clinical care will give rise to the development of virtual cohorts of cardiac models. These cohorts will allow cardiac simulations to capture and quantify inter-patient variability. However, the development of virtual cohorts of cardiac models will require the transformation of cardiac modelling from small numbers of bespoke models to robust and rapid workflows that can create large numbers of models. In this review, we describe the state of the art in virtual cohorts of cardiac models, the process of creating virtual cohorts of cardiac models, and how to generate the individual cohort member models, followed by a discussion of the potential and future applications of virtual cohorts of cardiac models.This article is part of the theme issue ‘Uncertainty quantification in cardiac and cardiovascular modelling and simulation’.

Computational modeling of cardiac electrophysiology (EP) has recently transitioned from a scientific research tool to clinical applications. To ensure reliability of clinical or regulatory decisions made using cardiac EP models, it is vital to evaluate the uncertainty in model predictions. Model predictions are uncertain because there is typically substantial uncertainty in model input parameters, due to measurement error or natural variability. While there has been much recent uncertainty quantification (UQ) research for cardiac EP models, all previous work has been limited by either: (i) considering uncertainty in only a subset of the full set of parameters; and/or (ii) assigning arbitrary variation to parameters (e.g., ±10 or 50% around mean value) rather than basing the parameter uncertainty on experimental data. In our recent work we overcame the first limitation by performing UQ and sensitivity analysis using a novel canine action potential model, allowing all parameters to be uncertain, but with arbitrary variation. Here, we address the second limitation by extending our previous work to use data-driven estimates of parameter uncertainty. Overall, we estimated uncertainty due to population variability in all parameters in five currents active during repolarization: inward potassium rectifier, transient outward potassium, L-type calcium, rapidly and slowly activating delayed potassium rectifier; 25 parameters in total (all model parameters except fast sodium current parameters). A variety of methods was used to estimate the variability in these parameters. We then propagated the uncertainties through the model to determine their impact on predictions of action potential shape, action potential duration (APD) prolongation due to drug block, and spiral wave dynamics. Parameter uncertainty had a significant effect on model predictions, especially L-type calcium current parameters. Correlation between physiological parameters was determined to play a role in physiological realism of action potentials. Surprisingly, even model outputs that were relative differences, specifically drug-induced APD prolongation, were heavily impacted by the underlying uncertainty. This is the first data-driven end-to-end UQ analysis in cardiac EP accounting for uncertainty in the vast majority of parameters, including first in tissue, and demonstrates how future UQ could be used to ensure model-based decisions are robust to all underlying parameter uncertainties.