This paper studies automated machine learning (AutoML) for causal discovery, the process of uncovering cause-and-effect relationships within data. Causal discovery is an unsupervised learning problem, as the target (the underlying ground truth causal model) is typically unknown. Therefore, the loss functions commonly used as an optimisation objective in AutoML systems developed for supervised learning problems are not applicable. We propose AutoCD, the first AutoML system utilising Bayesian optimisation based on a search space of causal discovery algorithms. In designing AutoCD, we study and compare the applicability of two different loss functions and post-hoc corrections. Additionally, based on the analysis of the performance of AutoCD, we propose an improved version called AutoCD(PC) by warm-starting the search from the PC algorithm. Results from our experiments on datasets simulated from 45 graphical models demonstrate that AutoCD PC performs better than the baselines by ranking the highest (avg. rank 3.69) compared to the best causal tuning baseline (avg. rank 5.21) and the best fine-tuned individual algorithm (avg. rank 4.36).
Many Bayesian Network structure learning algorithms are unstable in that the learnt graph is sensitive to arbitrary artefacts of the dataset, such as the ordering of columns (i.e., variable order). PC-Stable, developed by Colombo and Maathuis (2014), attempts to address this issue for the widely-used PC algorithm, prompting researchers to use the 'stable' version instead. However, this problem seems to have been overlooked for score-based algorithms. In this study, we show that some widely-used score-based algorithms suffer from the same issue and that PC-Stable, although less sensitive than most of the score-based algorithms tested, is not completely stable. We also present a solution to score-based greedy hill-climbing that completely eliminates this instability, and provide two implementations: the HC-Stable and Tabu-Stable algorithms, the latter of which learns more accurate graphs than all the well-known algorithms we compared it to.
A frequent goal in healthcare is to estimate personalized causal effects in order to select the best treatment for a patient from observational or experimental (RCT) data (or both), where "best" is defined in terms of maximizing the expectation of the desired outcome. The first task in estimating personalized effects is selecting the optimal set of personalization covariates (causal feature selection). This set of covariates is the Markov Boundary of the outcome in the experimental distribution, also known as the Interventional Markov Boundary (IMB), and can be identified from RCT data using methods for finding Markov Boundaries. However, most RCT data are very limited in sample size and do not work well with these methods. In this work, we develop methods that combine limited experimental and large observational data to identify the IMB, and improve the estimation of conditional (personalized) causal effects. These methods extend recent results (Triantafillou et al., 2021), which were limited to discrete data, to mixed data with binary and ordinal outcomes. The methods are based on Bayesian regression models. In simulated data, we show that our methods identify the correct IMB and improve causal effect estimation.
A common approach to learning Bayesian networks involves specifying an appropriately chosen family of parameterized probability density such as Gaussian. However, the distribution of most real-life data is leptokurtic and may not necessarily be best described by a Gaussian process. In this work we introduce Cauchy Graphical Models (CGM), a class of multivariate Cauchy densities that can be represented as directed acyclic graphs with arbitrary network topologies, the edges of which encode linear dependencies between random variables. We develop CGLearn, the resultant algorithm for learning the structure and Cauchy parameters based on Minimum Dispersion Criterion (MDC). Experiments using simulated datasets on benchmark network topologies demonstrate the efficacy of our approach when compared to Gaussian Graphical Models (GGM).
Bayesian inference in nonconjugate models such as Bayesian Poisson regression often relies on computationally expensive Monte Carlo methods. This paper introduces Q-conjugacy, a generalization of classical conjugacy that enables efficient closed-form variational inference in certain nonconjugate models. Q-conjugacy is a condition in which a closed-form update scheme expresses the solution minimizing the Kullback-Leibler divergence between a variational distribution and the product of two potentially unnormalized distributions. Leveraging Q-conjugacy within a local message passing framework allows deriving analytic inference update equations for nonconjugate models. The effectiveness of this approach is demonstrated on Bayesian Poisson regression and a model involving a hidden gamma-distributed latent variable with Gaussian-corrupted logarithmic observations. Results show that Q-conjugate triplets, such as (Gamma, LogNormal, Gamma), provide better speed-accuracy trade-offs than Markov Chain Monte Carlo.
Bayesian networks (BNs) are a popular framework in education and other fields. In this paper, we consider two-layer BNs, where the first layer consists of hidden binary variables that are assumed to be independent of each other, and the second layer consists of observed binary variables. The variables in the second layer depend on the variables in the first layer. The dependence is characterized by conditional probability tables, which represent Noisy-AND models. We refer to this class of models as BN2A models. We found that these models are also popular in the psychometric community, where they can be found under the name of Cognitive Diagnostic Models (CDMs), which are used to classify test takers into some latent classes according to the similarity of their responses to test questions. This paper shows the relation between some BN2A models and their corresponding CDMs. In particular, we compare the performance of these models on large-scale tests conducted in the Czech Republic in 2022. The BN2A model with general conditional probability tables produced the best absolute fit. However, when we added monotonic constraints to the General model, we obtained better predictive results.
Being a ubiquitous aspect of human cognition, causality has made its way into modern-day machine-learning research. Despite its importance in real-world applications, contemporary research still struggles with high-dimensional causal problems. Leveraging the efficiency of probabilistic circuits, which offer tractable computation of marginal probabilities, we introduce Psi net, a probabilistic model designed for large-scale causal inference. Psi net is a type of sum-product network where layering and the einsum operation allow for efficient parallelization. By incorporating interventional data into the learning process, the model can learn the effects of interventions and make predictions based on the specific interventional setting. Overall, Psi net is a causal probabilistic circuit that efficiently answers causal queries in large-scale problems. We present evaluations conducted on both synthetic data and a substantial real-world dataset, demonstrating.net's ability to capture causal relationships in high-dimensional settings.
Shared Decision Making (SDM) has become a predominant element of patient-centered healthcare delivery in recent years. In SDM, multiple agents, including a patient and a clinician interact to make a joint decision that is aligned with the patient's preferences. Despite its popularity, previous SDM studies lack structured decision modeling approaches applied to this problem. This paper presents Influence Diagram (ID) models for SDM agents, and proposes graphical operations for IDs to model the interaction between the agents. Using a case study, we demonstrate that widely used conceptual models for SDM such as the Three Talk Model are aligned with the proposed ID models and operations. The case study also shows that SDM is a cooperative decision making setting that is also present in non-clinical domains. The proposed influence diagrams and interaction operations enable SDM to be studied based on structured and quantitative decision models.
A Limited Memory Influence Diagram (LIMID) model for quality control that incorporates variable data on sample means from the output of a production process is introduced. The process operates over a finite production horizon and is out-of-control when the process mean for the output shifts. The probability such a shift occurs in the next time period is dependent on the elapsed time since the most recent process repair. A set of control limits that are adapted to the length the process has run without repair is selected to minimize quality control costs, and the sampling interval and sample size can be adjusted to further reduce costs if these modifications are operationally feasible. This is the first application of LIMIDs in a quality control model with an increasing rate of failure over time, and that implements variable data.
Fast arc-reversal (FAR) is proposed as a new exact inference algorithm in discrete Bayesian networks (BNs), merging favourable features of Arc-reversal (AR) and Variable elimination (VE). AR constantly maintains a sub-BN structure when rendering a variable barren via arc reversals, requiring more computational effort than VE, which sacrifices a sub-BN structure by directly eliminating a variable. We formally establish that FAR can recover a unique and sound sub-BN structure after consecutive variable eliminations. Experimental results on real-world benchmark networks empirically show a substantial improvement in the average run-time and variance of FAR compared to AR. We also suggest a novel method, called d-contraction, for graphically understanding FAR since FAR is not always the same as a sequence of arc reversals.
We enhance geometric Markov Chain Monte Carlo methods, in particular making them easier to use by providing better tools for choosing the metric and various tuning parameters. We extend the No-U-Turn criterion for automatic choice of integration length for Lagrangian Monte Carlo and propose a modification to the computationally efficient Monge metric, as well as summarizing several previously proposed metric choices. Through extensive experimentation, including synthetic examples and posteriordb benchmarks, we demonstrate that Riemannian metrics can outperform Euclidean counterparts, particularly in scenarios with high curvature, while highlighting how the optimal choice of metric is problem-specific.
The digital transformation of cities inspired the city administration of Aschaffenburg, Germany, to apply artificial intelligence to reduce the significant amount of manual administrative effort needed to evaluate citizens' ideas for potential future projects. This paper introduces a methodology that combines argument mining with Bayesian networks to evaluate the relative eligibility of city project proposals. The methodology involves two main steps: (1) clustering arguments extracted from public information available on the Internet, and (2) assessing and comparing selected urban issues, planning topics, and citizens' ideas that have been widely discussed to measure public interest in potential candidate projects. The results of the clustering are fed into a Bayesian network, along with scores for several evaluation criteria, to generate a relative eligibility score. The framework was applied to three candidate projects, resulting in the selection of one of them, while the other two were rejected with a given explanation. The latter motivates the decision and provides transparency to all parties involved in the decision process. The methodology is applicable to other cities after adjustments of criteria.
Structural causal models permit causal and counterfactual reasoning, and can be regarded as an extension of Bayesian networks. The model consists of endogenous and exogenous variables, with exogenous variables often being of unknown semantic interpretation. Consequently, they are typically non-observable, with the result that counterfactual queries may be unidentifiable. In this setting, standard inference algorithms for Bayesian networks are insufficient. Recent methods attempt to bound unidentifiable queries through imprecise estimation of exogenous probabilities. However, these approaches become unfeasible with growing cardinality of the exogenous variables. This paper proposes a divide and conquer method that transforms a general causal model into a set of models with low-cardinality exogenous variables, for which any query can be calculated exactly. Bounds for a query in the original model are then efficiently approximated by aggregating the results for the set of smaller models. Experimental results demonstrate that these bounds can be computed with lower error levels and less resource consumption compared to existing methods.
Counterfactual explanations are a very popular and effective method to convey interpretability in supervised classification models. These explanations answer the question of which change is needed in the input data to obtain a desired output. Computing good counterfactuals involves achieving some key objectives, such as validity, minimality, similarity or plausibility. Our proposal consists of using estimation of distribution algorithms for approximating counterfactual explanations within Bayesian classifiers. They are experimentally compared with a genetic algorithm, both with a single-objective and with a multi-objective formulation. Different types of Bayesian classifiers will be evaluated to find the differences in their explanations and we will use their results together to provide more accurate explanations. The experiments show how estimation of distribution algorithms are faster and achieve better results with a single-objective whereas they are competitive in the multi-objective version.
Recently, tensor networks have been proposed as a data structure for weighted model counting. Computing a weighted model count is thus reduced to contracting a factorized tensor expression. Inference queries on graphical models, especially PoE (probability of evidence) queries, can be expressed directly as weighted model counting problems. Maximization problems can also be addressed on the same data structure, only the standard sum-product semiring has to be replaced by either the tropical (max-sum) or the Viterbi (max-product) semiring in the computations, that is, the tensor contractions. However, tensor contractions only provide maximal values, but MPE (most probable explanation) queries on graphical models do not ask for the maximal value, but for a state, or even the states, at which the maximal value is attained. In the special case of tropical tensor networks for ground states of spin glasses, it has been observed that the ground state can be obtained by computing a derivative of the tensor network over the tropical semiring. Here, we generalize this observation, provide a generic algorithm for computing the derivatives, and prove its correctness.
Maximal ancestral graphs (MAGs) can represent causal relationships in systems that include unmeasured common direct causes. Constraint-based causal discovery methods are able to find solely the Markov Equivalence Class (MEC) of the causal structure given a set of observational data. To bound the total effect estimation between a pair of variables, when the MEC of the causal structure is known, the causal effect on each member in the MEC are computed, while keeping the minimum and maximum values as the lower and upper bounds for the total causal effect. However, when the modeling is done using MAGs, i.e., the MEC is encoded as a Partial Ancestral Graph (PAG), it is not always possible to find an adjustment set over some pairs of variables for the computation of the causal effect by covariance adjustment. In such cases, the LV-IDA algorithm returns missing values on the causal effects computation for some, and occasionally all, of the MAGs in the PAG. We present an extension of the LV-IDA algorithm, which we call the LV-IDA+ algorithm, that can compute approximated bounds of causal effects between every pair of the variables on a PAG. To achieve this, we propose a way to approximate the causal effect estimations when it is not possible to find adjustment sets for some pairs of variables on the MAGs in a PAG. We evaluate the performance of LV-IDA+ using simulated data generated by a canonical DAGs and compare with the LV-IDA algorithm. The results suggest the approximations of causal effects computed by LV-IDA+, are better than the missing values (simple NAs) returned by the LV-IDA algorithm, at least for the case of observational data generated by a canonical DAGs with latent variables.
Bayesian networks (BNs) represent a probabilistic model that can visualize relationships between variables. We apply various BN structure learning algorithms to a large dataset from a Czech university entrance exam. This dataset includes a test of active, open-minded thinking designed by Jonathan Baron, as well as a test of students' attitudes toward various conspiracies. Using BNs, we were able to identify the structure of the conspiracies and their relationships with active open-minded thinking. We also compared results of different BN structure learning algorithms with results of selected standard data analysis methods.
Bayesian network structure learning is an NP-hard problem. Furthermore, the problem remains hard even for various subclasses of graphs. Motivated by the hardness of exact learning, we study approximation algorithms for learning Bayesian networks. First, we propose a moderately exponential time algorithm with running time O(2(l/kn)) that has an approximation ratio l/k where n is the number of vertices and l and k are user-defined parameters with l <= k. That is, we give time-approximation trade-offs for learning Bayesian networks. Second, we present a polynomial time algorithm with an approximation ratio 1/d to find an optimal graph whose connected components have size at most d
In this paper, we address the use of the implicit hitting set approach (HS) for MAP (Markov Random Fields) and MPE (Bayesian Networks). Since the HS approach is quite general and finding the best version is very problem-dependent, here we present an adaptive algorithm that learns a reasonably good version for the instance being solved. The algorithm, which follows a Multi-armed Bandit structure, explores the different alternatives as it iterates and adapts their weights based on their performance. The weight is used to decide on the probability of selecting a given alternative in the next iteration.
