Machine learning for early syndrome diagnosis aims to solve the intricate task of predicting a ground truth label that most often is the outcome (effect) of a medical consensus definition applied to observed clinical measurements (causes), given clinical measurements observed several hours before. Instead of focusing on the prediction of the future effect, we propose to directly predict the causes via time series forecasting (TSF) of clinical variables and determine the effect by applying the gold standard consensus definition to the forecasted values. This method has the invaluable advantage of being straightforwardly interpretable to clinical practitioners, and because model training does not rely on a particular label anymore, the forecasted data can be used to predict any consensus-based label. We exemplify our method by means of long-term TSF with Transformer models, with a focus on accurate prediction of sparse clinical variables involved in the SOFA-based Sepsis-3 definition and the new Simplified Acute Physiology Score (SAPS-II) definition. Our experiments are conducted on two datasets and show that contrary to recent proposals which advocate set function encoders for time series and direct multi-step decoders, best results are achieved by a combination of standard dense encoders with iterative multi-step decoders. The key for success of iterative multi-step decoding can be attributed to its ability to capture cross-variate dependencies and to a student forcing training strategy that teaches the model to rely on its own previous time step predictions for the next time step prediction.

Empirical methods are means to answering methodological questions of empirical sciences by statistical techniques. The methodological questions addressed in this book include the problems of validity,

A restrictive fluid strategy is recommended in patients with acute respiratory distress syndrome (ARDS) managed with venovenous extracorporeal membrane oxygenation (VV ECMO). However, there are no established predictors for preload responsiveness in these patients. In 20 ARDS patients managed with VV ECMO, transesophageal echocardiography was used to repeatedly evaluate dynamic parameters of the left (velocity and stroke volume variation) and right ventricular outflow tract (velocity [respiratory variations of the maximal Doppler velocity in the truncus pulmonalis {ΔV max TP}] and velocity time integral [respiratory variation of the velocity time integral measured in the truncus pulmonalis {ΔVTI_TP}] variation in the truncus pulmonalis), the diameter variation in the superior and inferior vena cava and stroke volume variation measured by pulse contour analysis (SVV_PCA). Patients were categorized as responders and nonresponders according to an increase in stroke volume measured by echocardiography during a Passive Leg Raise Test with a cutoff value ≥10%. The final analysis includes 86 measurements. Predictive values for preload responsiveness were found for ΔV max TP (area under the curve [AUC] of 0.64), ΔVTI_TP (AUC 0.67), and SVV_PCA (AUC 0.74). In conclusion, SVV_PCA and, to a lesser extent, ΔV max TP and ΔVTI_TP are the most accurate parameters to predict preload responsiveness in ARDS patients managed with VV ECMO. Transesophageal echocardiography offers no advantages over pulse contour analysis for predicting preload responsiveness and provides only intermittent monitoring and assessment.

Reliability of machine learning evaluation -- the consistency of observed evaluation scores across replicated model training runs -- is affected by several sources of nondeterminism which can be regarded as measurement noise. Current tendencies to remove noise in order to enforce reproducibility of research results neglect inherent nondeterminism at the implementation level and disregard crucial interaction effects between algorithmic noise factors and data properties. This limits the scope of conclusions that can be drawn from such experiments. Instead of removing noise, we propose to incorporate several sources of variance, including their interaction with data properties, into an analysis of significance and reliability of machine learning evaluation, with the aim to draw inferences beyond particular instances of trained models. We show how to use linear mixed effects models (LMEMs) to analyze performance evaluation scores, and to conduct statistical inference with a generalized likelihood ratio test (GLRT). This allows us to incorporate arbitrary sources of noise like meta-parameter variations into statistical significance testing, and to assess performance differences conditional on data properties. Furthermore, a variance component analysis (VCA) enables the analysis of the contribution of noise sources to overall variance and the computation of a reliability coefficient by the ratio of substantial to total variance.

We demonstrate a validity problem of machine learning in the vital application area of disease diagnosis in medicine. It arises when target labels in training data are determined by an indirect measurement, and the fundamental measurements needed to determine this indirect measurement are included in the input data representation. Machine learning models trained on this data will learn nothing else but to exactly reconstruct the known target definition. Such models show perfect performance on similarly constructed test data but will fail catastrophically on real-world examples where the defining fundamental measurements are not or only incompletely available. We present a general procedure allowing identification of problematic datasets and black-box machine learning models trained on them, and exemplify our detection procedure on the task of early prediction of sepsis.

Ensembling neural networks is a long-standing technique for improving the generalization error of neural networks by combining networks with orthogonal properties via a committee decision. We show that this technique is an ideal fit for machine learning on medical data: First, ensembles are amenable to parallel and asynchronous learning, thus enabling efficient training of patient-specific component neural networks. Second, building on the idea of minimizing generalization error by selecting uncorrelated patient-specific networks, we show that one can build an ensemble of a few selected patient-specific models that outperforms a single model trained on much larger pooled datasets. Third, the non-iterative ensemble combination step is an optimal low-dimensional entry point to apply output perturbation to guarantee the privacy of the patient-specific networks. We exemplify our framework of differentially private ensembles on the task of early prediction of sepsis, using real-life intensive care unit data labeled by clinical experts.

Background Sepsis is the leading cause of death in the intensive care unit (ICU). Expediting its diagnosis, largely determined by clinical assessment, improves survival. Predictive and explanatory modelling of sepsis in the critically ill commonly bases both outcome definition and predictions on clinical criteria for consensus definitions of sepsis, leading to circularity. As a remedy, we collected ground truth labels for sepsis. Methods In the Ground Truth for Sepsis Questionnaire (GTSQ), senior attending physicians in the ICU documented daily their opinion on each patient’s condition regarding sepsis as a five-category working diagnosis and nine related items. Working diagnosis groups were described and compared and their SOFA-scores analyzed with a generalized linear mixed model. Agreement and discriminatory performance measures for clinical criteria of sepsis and GTSQ labels as reference class were derived. Results We analyzed 7291 questionnaires and 761 complete encounters from the first survey year. Editing rates for all items were > 90%, and responses were consistent with current understanding of critical illness pathophysiology, including sepsis pathogenesis. Interrater agreement for presence and absence of sepsis was almost perfect but only slight for suspected infection. ICU mortality was 19.5% in encounters with SIRS as the “worst” working diagnosis compared to 5.9% with sepsis and 5.9% with severe sepsis without differences in admission and maximum SOFA. Compared to sepsis , proportions of GTSQs with SIRS plus acute organ dysfunction were equal and macrocirculatory abnormalities higher (p < 0.0001). SIRS proportionally ranked above sepsis in daily assessment of illness severity (p < 0.0001) . Separate analyses of neurosurgical referrals revealed similar differences. Discriminatory performance of Sepsis-1/2 and Sepsis-3 compared to GTSQ labels was similar with sensitivities around 70% and specificities 92%. Essentially no difference between the prevalence of SIRS and SOFA ≥ 2 yielded sensitivities and specificities for detecting sepsis onset close to 55% and 83%, respectively. Conclusions GTSQ labels are a valid measure of sepsis in the ICU. They reveal suspicion of infection as an unclear clinical concept and refute an illness severity hierarchy in the SIRS-sepsis-severe sepsis spectrum. Ground truth challenges the accuracy of Sepsis-1/2 and Sepsis-3 in detecting sepsis onset. It is an indispensable intermediate step towards advancing diagnosis and therapy in the ICU and, potentially, other health care settings.

This paper is an excerpt of an early version of Chapter 2 of the book "Validity, Reliability, and Significance. Empirical Methods for NLP and Data Science", by Stefan Riezler and Michael Hagmann, published in December 2021 by Morgan Claypool. Please see the book's homepage at https://www.morganclaypoolpublishers.com/catalog_Orig/product_info.php?products_id=1688 for a more recent and comprehensive discussion.

Background Intestinal ischemia is a common complication with obscure pathophysiology in critically ill patients. Since insufficient delivery of oxygen is discussed, we investigated the influence of oxygen delivery, hemoglobin, arterial oxygen saturation, cardiac index and the systemic vascular resistance index on the development of intestinal ischemia. Furthermore, we evaluated the predictive power of elevated lactate levels for the diagnosis of intestinal ischemia. Methods In a retrospective case-control study data (mean oxygen delivery, minimum oxygen delivery, systemic vascular resistance index) of critical ill patients from 02/2009–07/2017 were analyzed using a proportional hazard model. General model fit and linearity were tested by likelihood ratio tests. The components of oxygen delivery (hemoglobin, arterial oxygen saturation and cardiac index) were individually tested in models. Results 59 out of 874 patients developed intestinal ischemia. A mean oxygen delivery less than 250ml/min/m 2 (LRT vs. null model: p = 0.018; LRT for non-linearity: p = 0.012) as well as a minimum oxygen delivery less than 400ml/min/m 2 (LRT vs null model: p = 0.016; LRT for linearity: p = 0.019) were associated with increased risk of the development of intestinal ischemia. We found no significant influence of hemoglobin, arterial oxygen saturation, cardiac index or systemic vascular resistance index. Receiver operating characteristics analysis for elevated lactate levels, pH, CO 2 and central venous saturation was poor with an area under the receiver operating characteristic of 0.5324, 0.52, 0.6017 and 0.6786. Conclusion There was a significant correlation for mean and minimum oxygen delivery with the incidence of intestinal ischemia for values below 250ml/min/m 2 respectively 400ml/min/m 2 . Neither hemoglobin, arterial oxygen saturation, cardiac index, systemic vascular resistance index nor elevated lactate levels could be identified as individual risk factors.

Empirical methods are means to answering methodological questions of empirical sciences by statistical techniques. The methodological questions addressed in this book include the problems of validity,

OBJECTIVES:The aim of this study was to establish quantitative CT (qCT) parameters for pathophysiological understanding and clinical use in patients with acute respiratory distress syndrome (ARDS). The most promising parameter is introduced.MATERIALS AND METHODS:28 intubated patients with ARDS obtained a conventional CT scan in end-expiratory breathhold within the first 48 hours after admission to intensive care unit (ICU). Following manual segmentation, 137 volume- and lung weight-associated qCT parameters were correlated with 71 clinical parameters such as blood gases, applied ventilation pressures, pulse contour cardiac output measurements and established status and prognosis scores (SOFA, SAPS II).RESULTS:Of all examined qCT parameters, excess lung weight (ELW), i.e. the difference between a patient's current lung weight and the virtual lung weight of a healthy person at the same height, displayed the most significant results. ELW correlated significantly with the amount of inflated lung tissue [%] (p<0.0001; r = -0.66) and was closely associated with the amount of extravascular lung water (EVLW) (p<0.0001; r = 0.72). More substantially than the oxygenation index (PaO2/FiO2) or any other clinical parameter it correlated with the patients' mean SOFA- (p<0.0001, r = 0.69) and SAPS II-Score (p = 0.0005, r = 0.62). Patients who did not survive intensive care treatment displayed higher values of ELW in the initial CT scans.CONCLUSIONS:ELW could serve as a non-invasive method to quantify the amount of pulmonary oedema. It might serve as an early radiological marker of severity in patients with ARDS.

Screw osteosynthesis using headless compression screws has become the accepted gold standard for the surgical treatment of scaphoid fractures. Optimal screw specifications remain controversially discussed. We aimed to investigate the influence of bone model composition on screw stability tests using headless compression screws in different scaphoid fracture models. We conducted pull-out tests using Acutrak2®mini, HCS®, HKS®, HBS®, Herbert/Whipple® and Twinfix® screws. To imitate cortical and cancellous bone, two-layer polyurethane (PU) models with two distinct densities were produced. The cylinders were cut at different positions to replicate fracture localisations at increasing distances. The maximum pull-out force required to achieve up to 1 mm of pull-out distance (Nto 1 mm) was measured. Acutrak2®mini and HCS® followed by Twinfix® showed the greatest average pull-out forces. Nto 1 mm was, on average, greater in the cortico-cancellous model than in the cancellous cylinder with the Acutrak2®mini and the Herbert/Whipple® screws, while it was the least with the HBS® and the Twinfix® screws; there were also differences between the HCS® and HKS®. There were no differences between the different fracture simulations in the synthesis strength using either the HKS® or HBS®. The pull-out forces of the HCS® and Twinfix® remained high also in simulations with the smaller screw base fragments. Varying imitations of cancellous and cortico-cancellous bone and fracture localisation reveal important information about the ex vivo strength of screw syntheses. The grip of the cortical structure should be used with the screws that fit more firmly in cortico-cancellous bone.

Background Sepsis-3 definition uses SOFA score to discriminate sepsis from uncomplicated infection, replacing SIRS criteria that were criticized for being inaccurate. Eligibility of sepsis-3 criteria for sepsis diagnosis and the applied validation methodology using mortality as endpoint are topic of ongoing debate. We assessed the impact of different criteria on sepsis diagnosis in our ICU and devised a mathematical approach for mortality-based validation of sepsis criteria. As infectious status is often unclear at clinical deterioration, we integrated non-infected patients into analysis. Methods Suspected infection, SOFA and SIRS were captured for an ICU cohort of a university center over one year. For raw scores (SIRS/SOFA) and sepsis criteria (SIRS≥2/SOFA≥2/SOFA_change≥2) frequencies and associations with in-hospital mortality were assessed. Using a mathematical approach, we estimated the correlation between sepsis and in-hospital mortality serving as reference for evaluation of observed mortality correlations of sepsis criteria. Results Of 791 patients, 369 (47%) were infected and 422 (53%) non-infected, with an in-hospital mortality of 39% and 15%. SIRS≥2 indicated sepsis in 90% of infected patients, SOFA≥2 in 99% and SOFA_change≥2 in 77%. In non-infected patients, SIRS, SOFA and SOFA_change were ≥2 in 78%, 88% and 58%. In AUROC analyses neither SOFA nor SIRS displayed superior mortality discrimination in infected compared to non-infected patients. The mathematically estimated correlation of sepsis and in-hospital mortality was 0.10 in infected and 0 in non-infected patients. Among sepsis criteria, solely SIRS≥2 agreed with expected correlations in both subgroups (infected: r = 0.19; non-infected: r = 0.02). Conclusions SOFA≥2 yielded a more liberal sepsis diagnosis than SIRS≥2. None of the criteria showed an infection specific occurrence that would be essential for reliable sepsis detection. However, SIRS≥2 matched the mortality association pattern of a valid sepsis criterion, whereas SOFA-based criteria did not. With this study, we establish a mathematical approach to mortality-based evaluation of sepsis criteria.