Deep neural networks have become state-of-the-art in medical image segmentation. However, the calibration of these models is an often overlooked aspect of the model’s performance, even though calibrated outputs communicate an intuitive measure of uncertainty toward the user. While other uncertainty measures have been applied in segmentation, work using existing post hoc calibration methods is lacking. In this paper, we investigated several post hoc calibration methods and introduced two straightforward extensions of Platt scaling and beta calibration that leverage spatial information available in the segmentation map. We compare these methods on the BraTS 2018, ISLES 2018, and QUBIQ datasets. On average, the fine-tuning method, isotonic regression method, and the extension of beta calibration performed the best calibration-wise: the Expected Calibration Error (ECE) decreased by 67.6

Low-voltage grid (LVG) state estimations help in expansion planning and preventing congestion events. However, country-scale simulations pose high computational burdens. A solution that reduces calculation time is to cluster similar days, and only simulate the most representative day of each cluster. Using real- world, quarter-hour residential consumption time series from 925 meters, congestion event probabilities from 146 real feeders, 51 daily meteorological and 12 calendar features, we propose a novel end-to-end representative-days clustering framework. Along with k-medoids clustering, we use dimensionality reduction (kernel principal component analysis, factor analysis, ... ) and pre/post-processing. To emphasize quarterhour extremes, we apply dynamic data squeezing/expansion, based on the LVG consumption median. Our approach is scalable because dimensionality reduction compresses thousands of daily variables into up to 300 components, regardless of the amount of meters and exogenous features (weather, calendar, ... ). Multi- objective Bayesian optimization for noisy functions, along with Sobol sampling, finds the hyperparameters that minimize the k representative-day reconstruction error compared to the full-year simulation. Explainability is achieved via tree ensemble classification on the decided clusters: ranking of the most important meteorological and calendar features, along with rule induction for future clustering decisions. We applied these techniques to successfully cluster all days of 2016 of the Flemish LVG (in Belgium). Our approach works well on both meter consumptions and feeder congestion event approximations. Photosynthetic radiation (visible spectrum and solar panel absorption wavelengths), water temperature, soil water volume/temperature (variables with a recent weather memory effect) and albedo were the most important meteorological factors, along with calendar features.

Electric vehicles (EVs), heat pumps (HPs), batteries and photovoltaic panels are being connected to the low-voltage grid (LVG) at a fast pace. The impact of these devices on the timing and size of LV feeder peaks is not well understood. A major hurdle has been the lack of smart meter data that contains a large amount of these devices. We recently investigated the impact on the timing and size of the feeder peak from 42,089 smart meter profiles of residential LVG customers in Flanders, Belgium. They were measured for a full year in 2022 by the distribution system operator Fluvius. In this paper, we present an updated and expanded analysis for smart meter profiles measured in 2023. The number of profiles has increased to 202,659. Batteries have been added as a category, and the number of profiles with both an EV and a HP is now large enough for statistical analysis. The size of the feeder peak is hardly influenced by batteries. A building with both an EV and a HP adds on average 2.3 kW to the feeder peak for a feeder with 10 connections, which is more than an EV (1.7 kW) or a HP (1.0 kW).

Accurate short-term prediction of individual residential load is essential for various applications and low-voltage grid stakeholders. Distribution system operators can utilize these forecasts in grid simulations, operation planning, and to anticipate unusual or varying consumption in certain neighborhoods, which can help to avoid congestions caused by peaks. Furthermore, these predictions can be used to optimize the low-voltage grid for renewable energy sources and potential battery storage. Due to the highly stochastic nature of household consumption, point forecasting is not optimal. We apply state-of-the-art probabilistic methods from other applications, and we propose a novel probabilistic global approach. It is based on a selection of similar normalized household consumption time series, which are then used to compute empirical quantiles. The proposed method has been evaluated on two different open datasets from different countries, and it outperforms state-of-the-art models in both cases. It has also been applied to a private Belgian dataset. In all three cases, the proposed method consistently outperforms the state-of-the-art methods on highly stochastic and changing households, i.e., households experiencing concept drift. This method is scalable, with low computational requirements, requires only seven days of historical data of the target household to make predictions and does not require household-specific or weather information.

Electric Vehicles (EVs), Heat Pumps (HPs) and solar panels are Low-Carbon Technologies (LCTs) that are being connected to the Low-Voltage Grid (LVG) at a rapid pace. One of the main hurdles to understand their impact on the LVG is the lack of recent, large electricity consumption datasets, measured in real-world conditions. We investigated the contribution of LCTs to the size and timing of peaks on LV feeders by using a large dataset of 42,089 smart meter profiles of residential LVG customers. These profiles were measured in 2022 by Fluvius, the Distribution System Operator (DSO) of Flanders, Belgium. The dataset contains customers that proactively requested higher-resolution smart metering data, and hence is biased towards energy-interested people. LV feeders of different sizes were statistically modeled with a profile sampling approach. For feeders with 40 connections, we found a contribution to the feeder peak of 1.2 kW for a HP, 1.4 kW for an EV and 2.0 kW for an EV charging faster than 6.5 kW. A visual analysis of the feeder-level loads shows that the classical duck curve is replaced by a night-camel curve for feeders with only HPs and a night-dromedary curve for feeders with only EVs charging faster than 6.5 kW. Consumption patterns will continue to change as the energy transition is carried out, because of e.g. dynamic electricity tariffs or increased battery capacities. Our introduced methods are simple to implement, making it a useful tool for DSOs that have access to smart meter data to monitor changing consumption patterns.

Precise calculations of network states and parameters are crucial for the modeling, monitoring, and operation of the distribution grid. AC power flow calculations are the preferred method for determining grid operating limits and assessing system hosting capacity. Traditional numerical power flow solvers are computationally intensive for large networks due to their iterative nature. In this context, we propose a physics-informed graph neural network technique as a fast alternative to the iterative power flow calculation methods. Our approach leverages the inherent physical knowledge of power systems through the integration of different physical constraints inside the model architecture and loss function. By incorporating physical laws such as Kirchhoff’s law and active power balance as prior knowledge, the proposed method maintains high accuracy while significantly reducing computational time. The results show that the proposed model achieves better performance and higher generalizability than off-the-shelf data-driven approaches.

Background Disability progression is a key milestone in the disease evolution of people with multiple sclerosis (PwMS). Prediction models of the probability of disability progression have not yet reached the level of trust needed to be adopted in the clinic. A common benchmark to assess model development in multiple sclerosis is also currently lacking. Methods Data of adult PwMS with a follow-up of at least three years from 146 MS centers, spread over 40 countries and collected by the MSBase consortium was used. With basic inclusion criteria for quality requirements, it represents a total of 15, 240 PwMS. External validation was performed and repeated five times to assess the significance of the results. Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) guidelines were followed. Confirmed disability progression after two years was predicted, with a confirmation window of six months. Only routinely collected variables were used such as the expanded disability status scale, treatment, relapse information, and MS course. To learn the probability of disability progression, state-of-the-art machine learning models were investigated. The discrimination performance of the models is evaluated with the area under the receiver operator curve (ROC-AUC) and under the precision recall curve (AUC-PR), and their calibration via the Brier score and the expected calibration error. All our preprocessing and model code are available at https://gitlab.com/edebrouwer/ms_benchmark, making this task an ideal benchmark for predicting disability progression in MS. Findings Machine learning models achieved a ROC-AUC of 0⋅71 ± 0⋅01, an AUC-PR of 0⋅26 ± 0⋅02, a Brier score of 0⋅1 ± 0⋅01 and an expected calibration error of 0⋅07 ± 0⋅04. The history of disability progression was identified as being more predictive for future disability progression than the treatment or relapses history. Conclusions Good discrimination and calibration performance on an external validation set is achieved, using only routinely collected variables. This suggests machine-learning models can reliably inform clinicians about the future occurrence of progression and are mature for a clinical impact study.

Feature attribution maps are a popular approach to highlight the most important pixels in an image for a given prediction of a model. Despite a recent growth in popularity and available methods, little attention is given to the objective evaluation of such attribution maps. Building on previous work in this domain, we investigate existing metrics and propose new variants of metrics for the evaluation of attribution maps. We confirm a recent finding that different attribution metrics seem to measure different underlying concepts of attribution maps, and extend this finding to a larger selection of attribution metrics. We also find that metric results on one dataset do not necessarily generalize to other datasets, and methods with desirable theoretical properties such as DeepSHAP do not necessarily outperform computationally cheaper alternatives. Based on these findings, we propose a general benchmarking approach to identify the ideal feature attribution method for a given use case. Implementations of attribution metrics and our experiments are available online.

The low-voltage grid (LVG) needs to be reinforced to handle the increased load due to the transition towards renewable energy. Doing this optimally requires knowledge of typical currents and voltages throughout the grid, which are unknown. They can be calculated from the grid layout and electricity consumption time series of each consumer, but for many consumers this time series is unknown. To alleviate this problem, we have developed two techniques to generate accurate and realistic daylong electricity consumption time series (scenarios) for a given consumer. Both techniques generate scenarios by sampling from historical consumption measurements of a limited set of consumers, considering available information about consumers (e.g., total yearly consumption) and days (e.g., weather). The first technique uses expert knowledge to define this sampling procedure, whereas the second learns it automatically using machine learning. The quality of the generated scenarios is evaluated by estimating how well the distributions of predicted and observed time series match, conditional on the available information. The data-driven technique performs better than the expert-based technique and, contrary to the latter, can easily be applied to datasets with different attributes without requiring any domain knowledge. Both proposed techniques outperform random sampling (the prevalent approach in existing LVG studies) and standard load profiles (commonly used by distribution system operators (DSOs)). Most of the improvement is obtained by including consumer attributes, whereas daily attributes lead to little or no performance improvement. The proposed techniques can help DSOs estimate the state of the LVG more accurately, thereby reducing investment costs.

Due to the increased penetration of low-carbon technologies, low voltage (LV) distribution networks are expected to face frequent congestion and voltage violation issues in the near future. In order to identify network limitations and make grid reinforcement decisions, comprehensive visibility of the LV network is needed. Inherently, LV grids are not well monitored, and most of the time the load profiles of the customers are not known. In this context, we present a two-stage novel scenario generation technique that combines a weight coefficients learning strategy with advanced clustering techniques to accurately sample optimal load profiles from the historical data for all the LV customers. Clustering based on Gaussian mixture models is used to extrapolate a limited number of measured yearlong load profiles to all customers without load profiles. A weighted dynamic time warping distance metric is then employed to group identical daylong load profiles into similar sub-clusters. Because different meteorological factors such as temperature, humidity, cloud cover, etc. affect consumption patterns differently under different conditions, a continuous genetic algorithm was developed to model the individual impact of different weather attributes on customer demand using weight coefficients. In addition, an unsupervised nearest neighbor search is incorporated to remove outliers and to ensure a certain degree of consistency among the selected scenarios. The proposed technique is evaluated using proper scoring metrics and it performs 40% better than the widely used random sampling and produces 15% more accurate results than the cluster sampling approach.

Short-term individual household load forecasting is relevant for several applications and low voltage grid (LVG) stakeholders, e.g., for grid simulations, operation planning, congestion anticipation or advance payments. Electrical consumption at the household level is highly stochastic, point forecasting cannot capture this efficiently. To have insights about the uncertainty of the prediction, probabilistic methods should be developed. We propose a method to predict the half-hourly consumption of individual households one day ahead, based on a neural network, enhanced with empirical quantiles based on the point forecasts errors. The method is scalable thanks to its low computational requirements. Additionally, it requires only historical data and calendar features. Finally, the method is evaluated in a case study where it achieves state-of-the-art accuracy.

: Data quality (DQ) measures data status based on different dimensions. This broad topic was brought to the fore in the ’80s when it was first discussed and studied. A high-quality dataset correlates with good performance in artificial intelligence (AI) algorithms and decision-making processes. Therefore, checking the quality of the data inside a decision support system (DSS) is an essential pre-processing step and is beneficial for improving further analysis. In this paper, a theoretical framework for a DQ module for a DSS is proposed. The framework evaluates the quality status in three stages: as based on the European guidelines, as based on DQ metrics, and as based on checking a subset of data cleaning (DC) problems. Additionally, the framework supports the user in identifying and fixing the DC problems, which speeds up the process. As output, the user receives a DQ report and the DC pipeline to execute to improve the dataset’s quality. An implementation of the framework is illustrated in a proof-of-concept (POC) for an industrial use case. In the POC, an example of the execution of the various framework phases was shown using a public time series dataset containing quarter-hourly consumption profiles of residential electricity customers in Belgium for the year 2016.

Multiple sclerosis (MS) is a chronic disease affecting millions of people worldwide. Through the demyelinating and axonal pathology of MS, the signal conduction in the central nervous system is affected. Evoked potential measurements allow clinicians to monitor this process and can be used for decision support. We share a dataset that contains motor evoked potential (MEP) measurements, in which the brain is stimulated and the resulting signal is measured in the hands and feet. This results in time series of 100 milliseconds long. Typically, both hands and feet are measured in one hospital visit. The dataset contains 5586 visits of 963 patients, performed in day-to-day clinical care over a period of 6 years. The dataset consists of approximately 100,000 MEP. Clinical metadata such as the expanded disability status scale, sex, and age is also available. This dataset can be used to explore the role of evoked potentials in MS research and patient care. It may also be used as a benchmark for time series analysis and predictive modelling.

Neural networks for automated image segmentation are typically trained to achieve maximum accuracy, while less attention has been given to the calibration of their confidence scores. However, well-calibrated confidence scores provide valuable information towards the user. We investigate several post hoc calibration methods that are straightforward to implement, some of which are novel. They are compared to Monte Carlo (MC) dropout. They are applied to neural networks trained with cross-entropy (CE) and soft Dice (SD) losses on BraTS 2018 and ISLES 2018. Surprisingly, models trained on SD loss are not necessarily less calibrated than those trained on CE loss. In all cases, at least one post hoc method improves the calibration. There is limited consistency across the results, so we can't conclude on one method being superior. In all cases, post hoc calibration is competitive with MC dropout. Although average calibration improves compared to the base model, subject-level variance of the calibration remains similar.

BACKGROUND AND OBJECTIVES:Research in Multiple Sclerosis (MS) has recently focused on extracting knowledge from real-world clinical data sources. This type of data is more abundant than data produced during clinical trials and potentially more informative about real-world clinical practice. However, this comes at the cost of less curated and controlled data sets. In this work we aim to predict disability progression by optimally extracting information from longitudinal patient data in the real-world setting, with a special focus on the sporadic sampling problem.METHODS:We use machine learning methods suited for patient trajectories modeling, such as recurrent neural networks and tensor factorization. A subset of 6682 patients from the MSBase registry is used.RESULTS:We can predict disability progression of patients in a two-year horizon with an ROC-AUC of 0.85, which represents a 32% decrease in the ranking pair error (1-AUC) compared to reference methods using static clinical features.CONCLUSIONS:Compared to the models available in the literature, this work uses the most complete patient history for MS disease progression prediction and represents a step forward towards AI-assisted precision medicine in MS.