This paper reviews recent works related to applications of reinforcement learning in power system optimal control problems. Based on an extensive analysis of works in the recent literature, we attempt to better understand the gap between reinforcement learning methods that rely on complete or incomplete information about the model dynamics and data-driven reinforcement learning approaches. More specifically we ask how such models change based on the application or the algorithm, what the currently open theoretical and numerical challenges are in each of the leading applications, and which reinforcement-based control strategies will rise in the following years. The reviewed research works are divided into “model-based” methods and “model-free” methods in order to highlight the current developments and trends within each of these two groups. The optimal control problems reviewed are energy markets, grid stability and control, energy management in buildings, electrical vehicles, and energy storage.

Forecasting errors in power markets, even as small as 1%, can have significant financial implications. However, even high-performance artificial intelligence (AI) based electricity price forecasting (EPF) models have instances when their prediction error is much higher than those shown by mean performance metrics. To date, explainable AI has been used to enhance the model transparency and trustworthiness of AI-based EPF models. However, this paper demonstrates that insights from explainable AI (XAI) techniques can be expanded beyond its primary task of explanatory visualizations. This work presents a XAI-based error compensation approach to improve model performance and identify irregular predictions. The first phase of the proposed approach involves error quantification through a Shapley additive explanations (SHAP) based corrector model that fine-tunes the base predictor’s forecasts. Using this corrector model’s SHAP explanations, the proposed approach distinguishes high-accuracy predictions from lower ones in the second stage. Additionally, these explanations are more simplified than the base model, making them easier for non-expert users like bidding agents. Performance enhancement and insightful user-centric explanations are crucial for real-world scenarios like price spikes during network congestion, high renewable penetration, and fluctuating fuel costs. Case studies discussed here show the efficacy of the proposed approach independent of model architecture, feature combination, or behavioral patterns of electricity prices in different markets.

The Colonoscopic Withdrawal Time (CWT) is the time required to withdraw the endoscope during a colonoscopy procedure. Estimating the CWT has several applications, including as a performance metric for gastroenterologists, and as an augmentation to polyp detection systems. We present a method for estimating the CWT directly from colonoscopy video based on three separate modules: egomotion computation; depth estimation; and anatomical landmark classification. Features are computed based on the modules' outputs, which are then used to classify each frame as representing forward, stagnant, or backward motion. This allows for the optimal detection of the change points between these phases based on efficient maximization of the likelihood; from which the CWT follows directly. We collect a dataset consisting of 788 videos of colonoscopy procedures, with the CWT for each annotated by gastroenterologists. Our algorithm achieves a mean error of 1.20 min, which nearly matches the inter-rater disagreement of 1.17 min.

The growing heterogenous ecosystem of networked consumer devices such as smart meters or IoT-connected appliances such as air conditioners is difficult to secure, unlike the utility side of the grid which can be defended effectively through rigorous IT security measures such as isolated control networks. In this paper, we consider a crisis scenario in which an attacker compromises a large number of consumer-side devices and modulates their electrical power to destabilize the grid and cause an electrical outage [9, 26, 27, 47, 50, 55]. In this paper propose a broadcast channel based on the modulation of grid frequency through which utility operators can issue commands to devices at the consumer premises both during an attack for mitigation and in its wake to aid recovery. Our proposed grid frequency modulation (GFM) channel is independent of other telecommunication networks. It is resilient towards localized blackouts and it is operational immediately after power is restored. Based on our GFM broadcast channel we propose a "safety reset" system to mitigate an ongoing attack by disabling a device's network interfaces and resetting its control functions. It can also be used in the wake of an attack to aid recovery by shutting down non-essential loads to reduce strain on the grid. To validate our proposed design, we conducted simulations based on measured grid frequency behavior. Based on these simulations, we performed an experimental validation on simulated grid voltage waveforms using a smart meter equipped with a prototype safety reset system based on a commodity microcontroller.

A service function chain defines an ordered or partially ordered set of abstract service functions and ordering constraints that must be applied to packet flows as a result of classification. Service chain verification is an important logic, which should verify that each flow indeed traverses the intended set of services, and in the desired order. In this paper we address this service chain verification problem in a new way. The main idea is to convert each service chain verification instance to its “equivalent set-expression cardinality equation,” and to use statistical algorithms to verify that each equation is satisfied.

BACKGROUND AND AIMS:Colorectal cancer is a leading cause of death. Colonoscopy is the criterion standard for detection and removal of precancerous lesions and has been shown to reduce mortality. The polyp miss rate during colonoscopies is 22% to 28%. DEEP DEtection of Elusive Polyps (DEEP2) is a new polyp detection system based on deep learning that alerts the operator in real time to the presence and location of polyps. The primary outcome was the performance of DEEP2 on the detection of elusive polyps. METHODS:The DEEP2 system was trained on 3611 hours of colonoscopy videos derived from 2 sources and was validated on a set comprising 1393 hours from a third unrelated source. Ground truth labeling was provided by offline gastroenterologist annotators who were able to watch the video in slow motion and pause and rewind as required. To assess applicability, stability, and user experience and to obtain some preliminary data on performance in a real-life scenario, a preliminary prospective clinical validation study was performed comprising 100 procedures. RESULTS:DEEP2 achieved a sensitivity of 97.1% at 4.6 false alarms per video for all polyps and of 88.5% and 84.9% for polyps in the field of view for less than 5 and 2 seconds, respectively. DEEP2 was able to detect polyps not seen by live real-time endoscopists or offline annotators in an average of .22 polyps per sequence. In the clinical validation study, the system detected an average of .89 additional polyps per procedure. No adverse events occurred. CONCLUSIONS:DEEP2 has a high sensitivity for polyp detection and was effective in increasing the detection of polyps both in colonoscopy videos and in real procedures with a low number of false alarms. (Clinical trial registration number: NCT04693078.).

A challenging open question in deep learning is how to handle tabular data. Unlike domains such as image and natural language processing, where deep architectures prevail, there is still no widely accepted neural architecture that dominates tabular data. As a step toward bridging this gap, we present Net-DNF a novel generic architecture whose inductive bias elicits models whose structure corresponds to logical Boolean formulas in disjunctive normal form (DNF) over affine soft-threshold decision terms. Net-DNFs also promote localized decisions that are taken over small subsets of the features. We present an extensive experiments showing that Net-DNFs significantly and consistently outperform fully connected networks over tabular data. With relatively few hyperparameters, Net-DNFs open the door to practical end-to-end handling of tabular data using neural networks. We present ablation studies, which justify the design choices of Net-DNF including the inductive bias elements, namely, Boolean formulation, locality, and feature selection.

There is a striking relationship between a three hundred years old Political Science theorem named "Condorcet's jury theorem" (1785), which states that majorities are more likely to choose correctly when individual votes are often correct and independent, and a modern Machine Learning concept called "Strength of Weak Learnability" (1990), which describes a method for converting a weak learning algorithm into one that achieves arbitrarily high accuracy and stands in the basis of Ensemble Learning. Albeit the intuitive statement of Condorcet's theorem, we could not find a compact and simple rigorous mathematical proof of the theorem neither in classical handbooks of Machine Learning nor in published papers. By all means we do not claim to discover or reinvent a theory nor a result. We humbly want to offer a more publicly available simple derivation of the theorem. We will find joy in seeing more teachers of introduction-to-machine-learning courses use the proof we provide here as an exercise to explain the motivation of ensemble learning.

Colonoscopy is tool of choice for preventing Colorectal Cancer, by detecting and removing polyps before they become cancerous. However, colonoscopy is hampered by the fact that endoscopists routinely miss 22-28% of polyps. While some of these missed polyps appear in the endoscopist’s field of view, others are missed simply because of substandard coverage of the procedure, i.e. not all of the colon is seen. This paper attempts to rectify the problem of substandard coverage in colonoscopy through the introduction of the C2D2 (Colonoscopy Coverage Deficiency via Depth) algorithm which detects deficient coverage, and can thereby alert the endoscopist to revisit a given area. More specifically, C2D2 consists of two separate algorithms: the first performs depth estimation of the colon given an ordinary RGB video stream; while the second computes coverage given these depth estimates. Rather than compute coverage for the entire colon, our algorithm computes coverage locally, on a segment-by-segment basis; C2D2 can then indicate in real-time whether a particular area of the colon has suffered from deficient coverage, and if so the endoscopist can return to that area. Our coverage algorithm is the first such algorithm to be evaluated in a large-scale way; while our depth estimation technique is the first calibration-free unsupervised method applied to colonoscopies. The C2D2 algorithm achieves state of the art results in the detection of deficient coverage. On synthetic sequences with ground truth, it is 2.4 times more accurate than human experts; while on real sequences, C2D2 achieves a 93.0% agreement with experts.

We consider interactive proofs for social graphs, where the verifier has only oracle access to the graph and can query for the \(i^{th}\) neighbor of a vertex v, given i and v. In this model, we construct a doubly-efficient public-coin two-message interactive protocol for estimating the size of the graph to within a multiplicative factor \(\varepsilon >0\). The verifier performs \(\widetilde{O}(1/\varepsilon ^2 \cdot \tau _{mix}\cdot \varDelta )\) queries to the graph, where \(\tau _{mix}\) is the mixing time of the graph and \(\varDelta \) is the average degree of the graph. The prover runs in quasi-linear time in the number of nodes in the graph.

A challenging open question in deep learning is how to handle tabular data. Unlike domains such as image and natural language processing, where deep architectures prevail, there is still no widely accepted neural architecture that dominates tabular data. As a step toward bridging this gap, we present DNF-Net a novel generic architecture whose inductive bias elicits models whose structure corresponds to logical Boolean formulas in disjunctive normal form (DNF) over affine soft-threshold decision terms. In addition, DNF-Net promotes localized decisions that are taken over small subsets of the features. We present an extensive empirical study showing that DNF-Nets significantly and consistently outperform FCNs over tabular data. With relatively few hyperparameters, DNF-Nets open the door to practical end-to-end handling of tabular data using neural networks. We present ablation studies, which justify the design choices of DNF-Net including the three inductive bias elements, namely, Boolean formulation, locality, and feature selection.

Deep neural models, such as convolutional and recurrent networks, achieve phenomenal results over spatial data such as images and text. However, when considering tabular data, gradient boosting of decision trees (GBDT) remains the method of choice. Aiming to bridge this gap, we propose \emph{deep neural forests} (DNF) -- a novel architecture that combines elements from decision trees as well as dense residual connections. We present the results of extensive empirical study in which we examine the performance of GBDTs, DNFs and (deep) fully-connected networks. These results indicate that DNFs achieve comparable results to GBDTs on tabular data, and open the door to end-to-end neural modeling of multi-modal data. To this end, we present a successful application of DNFs as part of a hybrid architecture for a multi-modal driving scene understanding classification task.

In this study, a new real-time optimisation method for reactive power distribution in microgrids is proposed. The method enables location of a globally optimal distribution of reactive power under normal operating conditions. The method exploits the typical compact structure of microgrids to obtain a solution by parts, using the dynamic programming method and Bellman equation. The proposed solution method is based on the fact that the microgrid is designed with a central feeder line to which clusters of generators and loads are connected, and is suitable for microgrids with ring topologies as well as radial ones. The optimisation problem is formulated with the cluster reactive powers as free variables, and the solution space is spanned by the cluster reactive power outputs. The optimal solution is then constructed by efficiently scanning the entire solution space, by testing every possible combination of reactive powers, using dynamic programming. Since every single step involves a one-dimensional problem, the complexity of the solution is only linear with the number of clusters, and as a result, a globally optimal solution may be obtained in real time. The study includes the results of two test-case networks.

In recent years there has been a growing interest in developing "streaming algorithms" for efficient processing and querying of continuous data streams. These algorithms seek to provide accurate results while minimizing the required storage and the processing time, at the price of a small inaccuracy in their output. A fundamental query of interest is the intersection size of two big data streams. This problem arises in many different application areas, such as network monitoring, database systems, data integration and information retrieval. In this paper we develop a new algorithm for this problem, based on the Maximum Likelihood (ML) method. We show that this algorithm outperforms all known schemes in terms of the estimation's quality (lower variance) and that it asymptotically achieves the optimal variance.

Cardinality estimation algorithms receive a stream of elements whose order might be arbitrary, with possible repetitions, and return the number of distinct elements. Such algorithms usually seek to minimize the required storage and processing at the price of inaccuracy in their output. Real-world applications of these algorithms are required to process large volumes of monitored data, making it impractical to collect and analyze the entire input stream. In such cases, it is common practice to sample and process only a small part of the stream elements. This paper presents and analyzes a generic algorithm for combining every cardinality estimation algorithm with a sampling process. We show that the proposed sampling algorithm does not affect the estimator's asymptotic unbiasedness, and we analyze the sampling effect on the estimator's variance.

Sketch-based streaming algorithms allow efficient processing of big data. These algorithms use small fixed-size storage to store a summary ("sketch") of the input data, and use probabilistic algorithms to estimate the desired quantity. However, in many real-world applications it is impractical to collect and process the entire data stream, the common practice is thus to sample and process only a small part of it. While sampling is crucial for handling massive data sets, it may reduce accuracy. In this paper we present a new framework that can accurately estimate the cardinality of any set expression between any number of streams using only a small sample of each stream. The proposed framework consists of a new sketch, called Maximal-Term with Subsample (MTS), and a family of algorithms that use this sketch. An example of a possible query that can be efficiently answered using the proposed sketch is, How many distinct tuples appear in tables $T_1$ and $T_2$, but not in $T_3$? The algorithms presented in this paper answer such queries accurately, processing only a small sample of the tuples in each table and using a constant amount of memory. Such estimations are useful for the optimization of queries over very large database systems. We show that all our algorithms are unbiased, and we analyze their asymptotic variance.

A high-voltage power supply for X-ray applications is proposed. In the proposed topology, a direct current input of 150-400 V is switched at high frequency (100 kHz) to generate multiphase quasi-sinusoidal voltages. Each phase is then fed to a half-wave Cockcroft-Walton voltage multiplier section through an isolation transformer, and the multiplier section outputs feed the load. The proposed topology boosts the voltage by two mechanisms: 1) the voltage multiplication and 2) the transformer-like mechanism attained by the multiple isolated sections. The proposed split-source multisection topology attains up to m times higher voltage gain and ripple mitigation (m being the splitting degree). Simulation and experimental results support the effectiveness of the proposed topology. For instance, in one of the experiments, the load voltage was increased from 2.8 to 6.4 kV by applying a split-source three-section topology.

High voltages are typically generated by voltage multipliers that consist of several multiplying stages, where each stage is composed of a few capacitors and diodes. The Half-Wave Cockcroft-Walton voltage multiplier is probably the most common multiplier. We compare different methods of capacitance distribution in the Half-Wave Cockcroft-Walton voltage multiplier and show that the output impedance can be decreased by more than three times compared to the case of the most commonly used method, i.e., the equal capacitance throughout all stages, and a higher output voltage is consequently gained (while yet using an equal total capacitance). For example, our experiment displayed a voltage increase from 5.8 kV to 9.7 kV across the load when the capacitance distributions were changed and the voltage multiplier was split into two sections.

A new high-voltage (HV) power supply based on the half-wave Cockcroft-Walton voltage multiplier topology is introduced. The proposed power supply operates at 1 MHz (at least ten times faster than conventional HV multipliers) and exhibits a very stable output voltage with minimal ripple (<;0.1%). The proposed power supply has a high power density due to the high frequency and is also highly efficient (>90%). The high operation frequency and efficiency were facilitated by a critical design that applies softly switched wide-bandgap devices (silicon carbide diodes and gallium nitride high electron mobility transistor cascode MOSFETS) in full bridge cascaded by a voltage multiplier topology. Experimental evidence is provided from a prototype unit that raises the voltage from 300 V to 5 kV, with a 93% efficiency.

In this paper we present a novel topology for high voltage generation based on the widely used Half-Wave Cockcroft-Walton voltage multiplier. The proposed topology consists of several voltage multiplying sections fed separately at their inputs and whose output voltages are summated across the load. Using the proposed topology we can attain over m times higher voltage gain compared to the regular Half-Wave Cockcroft Walton voltage multiplier (m being the splitting level), while significantly reducing the output ripple. The theory is supported by simulations and experiments.