Several explainable AI algorithms have been proposed to help make machine learning models more interpretable and trustworthy. However in spite of numerous methodological advancements, there is still a persistent gap between what researchers develop and what business users seek. In this work, we aim to bridge this gap for an AI system that predicts the remaining useful life of an aircraft’s engine using time series data collected from multiple sensors. We propose a novel approach to compute easily understandable explanations by fusing two explainers in sequence wherein explanations of the first explainer are explained by the second. We use this approach to build a global post-hoc model-agnostic explainer for AI models that ingest multivariate time series data. Our approach fuses a local explainer that yields feature importance weights, with a directly interpretable model that outputs global rules. Our experimental results based on the C-MAPSS open-source dataset demonstrate that the proposed two-stage explainer computes global explanations that are amenable to business users and sheds light on how the behavior of an individual and a group of sensors impacts the remaining useful life of an aircraft’s engine.

Locally interpretable model agnostic explanations (LIME) method is one of the most popular methods used to explain black-box models at a per example level. Although many variants have been proposed, few provide a simple way to produce high fidelity explanations that are also stable and intuitive. In this work, we provide a novel perspective by proposing a model agnostic local explanation method inspired by the invariant risk minimization (IRM) principle -- originally proposed for (global) out-of-distribution generalization -- to provide such high fidelity explanations that are also stable and unidirectional across nearby examples. Our method is based on a game theoretic formulation where we theoretically show that our approach has a strong tendency to eliminate features where the gradient of the black-box function abruptly changes sign in the locality of the example we want to explain, while in other cases it is more careful and will choose a more conservative (feature) attribution, a behavior which can be highly desirable for recourse. Empirically, we show on tabular, image and text data that the quality of our explanations with neighborhoods formed using random perturbations are much better than LIME and in some cases even comparable to other methods that use realistic neighbors sampled from the data manifold. This is desirable given that learning a manifold to either create realistic neighbors or to project explanations is typically expensive or may even be impossible. Moreover, our algorithm is simple and efficient to train, and can ascertain stable input features for local decisions of a black-box without access to side information such as a (partial) causal graph as has been seen in some recent works.

Locally interpretable model agnostic explanations (LIME) method is one of the most popular methods used to explain black-box models at a per example level. Although many variants have been proposed, few provide a simple way to produce high fidelity explanations that are also stable and intuitive. In this work, we provide a novel perspective by proposing a model agnostic local explanation method inspired by the invariant risk minimization (IRM) principle -- originally proposed for (global) out-of-distribution generalization -- to provide such high fidelity explanations that are also stable and unidirectional across nearby examples. Our method is based on a game theoretic formulation where we theoretically show that our approach has a strong tendency to eliminate features where the gradient of the black-box function abruptly changes sign in the locality of the example we want to explain, while in other cases it is more careful and will choose a more conservative (feature) attribution, a behavior which can be highly desirable for recourse. Empirically, we show on tabular, image and text data that the quality of our explanations with neighborhoods formed using random perturbations are much better than LIME and in some cases even comparable to other methods that use realistic neighbors sampled from the data manifold, where the latter is a popular strategy to obtain high quality explanations. This is a desirable property given that learning a manifold to either create realistic neighbors or to project explanations is typically expensive or may even be impossible. Moreover, our algorithm is simple and efficient to train, and can ascertain stable input features for local decisions of a black-box without access to side information such as a (partial) causal graph as has been seen in some recent works.

As artificial intelligence and machine learning algorithms become increasingly prevalent in society, multiple stakeholders are calling for these algorithms to provide explanations. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, have different explanation needs. To address these needs, in 2019, we created AI Explainability 360, an open source software toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics. This paper examines the impact of the toolkit with several case studies, statistics, and community feedback. The different ways in which users have experienced AI Explainability 360 have resulted in multiple types of impact and improvements in multiple metrics, highlighted by the adoption of the toolkit by the independent LF AI & Data Foundation. The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.

The loosely-coupled microservices architecture has become increasingly popular due to the advantage of its modularity and elasticity in cloud applications. However, it also seriously complicates cloud management and degrades the performance of IT operations. Today, AI has been the locus of commerce and transactions, and transforming traditional IT operations for speed and growth. Inferring the dependencies among an application's microservices can greatly help SREs diagnose possible root causes of performance issues, which is a hard task due to the complex topology of microservices is often unknown in practice. Prior literature on detecting causal structure for cloud services requires significant application instrumentation, which rarely holds in reality. In this work, we leverage Granger causality models on just monitored log data of a microservice-based application to infer the impact of dependencies between microservices. We first describe the approach of modeling discrete log data as time series, and then formally define the Granger causality problem using both linear and nonlinear autoregressive models. Finally, we conduct an extensive comparative study to show the performance of the state-of-the-art linear and nonlinear (i.e., neural) Granger causality methods on both synthetic data and real-world log data from a publicly available benchmark microservice system. Our preliminary results indicate that neural Granger causality models outperform traditional Granger causality methods on both linear and nonlinear time series data, while for large linear time series, linear Granger causal models are more efficient with high accuracy. Using the real-world log data, we also demonstrate our interesting findings on inferred dependency graph of microservices by linear and neural Granger causality models.

Inferring causality of events from log data is critical to IT operations teams who continuously strive to identify probable root causes of events in order to quickly resolve incident tickets so that downtimes and service interruptions are kept to a minimum. Although prior work has applied some specific causal inference techniques on proprietary log data, they fail to benchmark the performance of different techniques on a common system or dataset. In this work, we evaluate the performance of multiple state-of-the-art causal inference techniques using log data obtained from a publicly available benchmark microservice system. We model log data both as a timeseries of error counts and as a temporal event sequence and evaluate 3 families of Granger causal techniques: regression based, independence testing based, and event models. Our preliminary results indicate that event models yield causal graphs that have high precision and recall in comparison to regression and independence testing based Granger methods.

This tutorial will teach participants to use and contribute to a new open-source Python package named AI Explainability 360 (AIX360) (https://aix360.mybluemix.net), a comprehensive and extensible toolkit that supports interpretability and explainability of data and machine learning models. Motivation for the toolkit. The AIX360 toolkit illustrates that there is no single approach to explainability that works best for all situations. There are many ways to explain: data vs. model, direct vs. post-hoc explanation, local vs. global, etc. The toolkit includes ten state of the art algorithms that cover different dimensions of explanations along with proxy explainability metrics. Moreover, one of our prime objectives is for AIX360 to serve as an educational tool even for non-machine learning experts (viz. social scientists, healthcare experts). To this end, the toolkit has an interactive demonstration, highly descriptive Jupyter notebooks covering diverse real-world use cases, and guidance materials, all helping one navigate the complex explainability space. Compared to existing open-source efforts on AI explainability, AIX360 takes a step forward in focusing on a greater diversity of ways of explaining, usability in industry, and software engineering. By integrating these three aspects, we hope that AIX360 will attract researchers in AI explainability and help translate our collective research results for practicing data scientists and developers deploying solutions in a variety of industries. Regarding the first aspect of diversity, Table 1 in [1] compares AIX360 to existing toolkits in terms of the types of explainability methods offered. The table shows that AIX360 not only covers more types of methods but also has metrics which can act as proxies for judging the quality of explanations. Regarding the second aspect of industry usage, AIX360 illustrates how these explainability algorithms can be applied in specific contexts (please see Audience, goals, and outcomes below). In just a few months since its initial release, the AIX360 toolkit already has a vibrant slack community with over 120 members and has been forked almost 80 times accumulating over 400 stars. This response leads us to believe that there is significant interest in the community in learning more about the toolkit and explainability in general. Audience, goals, and outcomes. The presentations in the tutorial will be aimed at an audience with different backgrounds and computer science expertise levels. For all audience members and especially those unfamiliar with Python programming, AIX360 provides an interactive experience (http://aix360.mybluemix.net/data) centered around a credit approval scenario as a gentle and grounded introduction to the concepts and capabilities of the toolkit. We will also teach all participants which type of explainability algorithm is most appropriate for a given use case, not only for those in the toolkit but also from the broader explainability literature. Knowing which explainability algorithms apply to which contexts and understanding when to use them can benefit most people, regardless of their technical background. The second part of the tutorial will consist of three use cases featuring different industry domains and explanation methods. Data scientists and developers can gain hands-on experience with the toolkit by running and modifying Jupyter notebooks, while others will be able to follow along by viewing rendered versions of the notebooks. Here is a rough agenda of the tutorial: 1) Overture: Provide a brief introduction to the area of explainability as well as introduce common terms. 2) Interactive Web Experience: The AIX360 interactive web experience (http://aix360.mybluemix.net/data) is intended to show a non-computer science audience how different explainability methods may suit different stakeholders in a credit approval scenario (data scientists, loan officers, and bank customers). 3) Taxonomy: We will next present a taxonomy that we have created for organizing the space of explanations and guiding practitioners toward an appropriate choice for their applications. 4) Installation: We will transition into a Python environment and ask participants to install the AIX360 package on their machines using provided instructions. 5) Example Use Cases in Finance, Government, and Healthcare: We will take participants through three use-cases in various application domains in the form of Jupyter notebooks. 6) Metrics: We will briefly showcase the two explainability metrics currently available through the toolkit. 7) Future Directions: The final segment will be to discuss future directions and how participants can contribute to the toolkit.

As artificial intelligence algorithms make further inroads in high-stakes societal applications, there are increasing calls from multiple stakeholders for these algorithms to explain their outputs. To make matters more challenging, different personas of consumers of explanations have different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360, an open-source Python toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics (http://aix360.mybluemix.net). Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of interpretation and explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. The toolkit is not only the software, but also guidance material, tutorials, and an interactive web demo to introduce AI explainability to different audiences. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.

To preserve anonymity and obfuscate their identity on online platforms users may morph their text and portray themselves as a different gender or demographic. Similarly, a chatbot may need to customize its communication style to improve engagement with its audience. This manner of changing the style of written text has gained significant attention in recent years. Yet these past research works largely cater to the transfer of single style attributes. The disadvantage of focusing on a single style alone is that this often results in target text where other existing style attributes behave unpredictably or are unfairly dominated by the new style. To counteract this behavior, it would be nice to have a style transfer mechanism that can transfer or control multiple styles simultaneously and fairly. Through such an approach, one could obtain obfuscated or written text incorporated with a desired degree of multiple soft styles such as female-quality, politeness, or formalness. To the best of our knowledge this work is the first that shows and attempt to solve the issues related to multiple style transfer. We also demonstrate that the transfer of multiple styles cannot be achieved by sequentially performing multiple single-style transfers. This is because each single style-transfer step often reverses or dominates over the style incorporated by a previous transfer step. We then propose a neural network architecture for fairly transferring multiple style attributes in a given text. We test our architecture on the Yelp dataset to demonstrate our superior performance as compared to existing one-style transfer steps performed in a sequence.

There has been an unprecedented surge in the number of service providers offering a wide range of machine learning prediction APIs for tasks such as image classification, language translation, etc. thereby monetizing the underlying data and trained models. Typically, a data owner (API provider) develops a model, often over proprietary data, and leverages the infrastructure services of a cloud vendor for hosting and serving API requests. Clearly, this model assumes complete trust between the API Provider and cloud vendor. On the other hand, a malicious/buggy cloud vendor may copy the APIs and offer an identical service, under-report model usage metrics, or unfairly discriminate between different API providers by offering them a nominal share of the revenue. In this work, we present the design of a blockchain based decentralized trustless API marketplace that enables all the stakeholders in the API ecosystem to audit the behavior of the parties without having to trust a single centralized entity. In particular, our system divides an AI model into multiple pieces and deploys them among multiple cloud vendors who then collaboratively execute the APIs. Our design ensures that cloud vendors cannot collude with each other to steal the combined model, while individual cloud vendors and clients cannot repudiate their input or model executions.

We present a blockchain based system that allows data owners, cloud vendors, and AI developers to collaboratively train machine learning models in a trustless AI marketplace. Data is a highly valued digital asset and central to deriving business insights. Our system enables data owners to retain ownership and privacy of their data, while still allowing AI developers to leverage the data for training. Similarly, AI developers can utilize compute resources from cloud vendors without loosing ownership or privacy of their trained models. Our system protocols are set up to incentivize all three entities - data owners, cloud vendors, and AI developers to truthfully record their actions on the distributed ledger, so that the blockchain system provides verifiable evidence of wrongdoing and dispute resolution. Our system is implemented on the Hyperledger Fabric and can provide a viable alternative to centralized AI systems that do not guarantee data or model privacy. We present experimental performance results that demonstrate the latency and throughput of its transactions under different network configurations where peers on the blockchain may be spread across different datacenters and geographies. Our results indicate that the proposed solution scales well to large number of data and model owners and can train up to 70 models per second on a 12-peer non optimized blockchain network and roughly 30 models per second in a 24 peer network.

Condition monitoring and forecasting applications require accurate PV models that can predict power from weather parameters. However PV output is also dependent on the potentially suboptimal behavior of the MPPT controller, which can introduce both inefficiencies and prediction challenges. In this work, we use a data-driven approach to show that MPPT controllers do not always operate at the optimal knee point of the I-V curve and propose methods to quantify these inefficiencies. Based on these findings, we develop novel machine learning PV models that predict current and voltage separately and capture the behavior of the MPPT system more accurately. We present evaluation results using data collected from a large solar farm, which shows that the proposed models can reduce estimation errors significantly as compared to state of the art methods.

Energy costs from Heating, Ventilation and Air Conditioning (HVAC) systems can be greater than 50% of the total operational expenditure of a commercial building. Demand side strategies such as pre-cooling and supply side strategies such as using renewable energy resources (e.g. solar panels) have the potential to reduce the operating energy costs. Several optimization frameworks have been proposed that use these strategies in an efficient manner. However, incorporating the dynamics of HVAC in an optimization framework often renders the framework computationally intractable — due to the inherent non-linearities — for large buildings. In this paper, we focus on the demand side strategy and develop a closed form expression for the optimal pre-cooling time assuming a 1R, 1C model of a building. Using simulations, we demonstrate that the optimal pre-cooling time obtained from the closed form expression matches the one from the optimization framework. Furthermore, we show that the expression — when used in conjunction with existing HVACoptimization frameworks to determine the optimal schedule of set-point temperatures — reduces the computation time of the optimization by two orders of magnitude, without loss of optimality.

Machine learning models deployed on the cloud are susceptible to several security threats including extraction attacks. Adversaries may abuse a model's prediction API to steal the model thus compromising model confidentiality, privacy of training data, and revenue from future query payments. This work introduces a model extraction monitor that quantifies the extraction status of models by continually observing the API query and response streams of users. We present two novel strategies that measure either the information gain or the coverage of the feature space spanned by user queries to estimate the learning rate of individual and colluding adversaries. Both approaches have low computational overhead and can easily be offered as services to model owners to warn them against state of the art extraction attacks. We demonstrate empirical performance results of these approaches for decision tree and neural network models using open source datasets and BigML MLaaS platform.

One of the major challenges confronting the widespread adoption of solar energy is the uncertainty of production. The energy generated by photo-voltaic systems is a function of the received solar irradiance which varies due to atmospheric and weather conditions. A key component required for forecasting irradiance accurately is the clear sky model which estimates the average irradiance at a location at a given time in the absence of clouds. Current methods for modelling clear sky irradiance are either inaccurate or require extensive atmospheric data, which tends to vary with location and is often unavailable. In this paper, we present a data-driven methodology, Blue Skies, for modelling clear sky irradiance solely based on historical irradiance measurements. Using machine learning techniques, Blue Skies is able to generate clear sky models that are more accurate spatio-temporally compared to the state of the art, reducing errors by almost 50%.

In electricity markets, the choice of the right pricing regime is crucial for the utilities because the price they charge to their consumers, in anticipation of their demand in real-time, is a key determinant of their profits and ultimately their survival in competitive energy markets. Among the existing pricing regimes, in this paper, we consider ex-ante dynamic pricing schemes as (i) they help to address the peak demand problem (a crucial problem in smart grids), and (ii) they are transparent and fair to consumers as the cost of electricity can be calculated before the actual consumption. In particular, we propose an axiomatic framework that establishes the conceptual underpinnings of the class of ex-ante dynamic pricing schemes. We first propose five key axioms that reflect the criteria that are vital for energy utilities and their relationship with consumers. We then prove an impossibility theorem to show that there is no pricing regime that satisfies all the five axioms simultaneously. We also study multiple cost functions arising from various pricing regimes to examine the subset of axioms that they satisfy. We believe that our proposed framework in this paper is first of its kind to evaluate the class of ex-ante dynamic pricing schemes in a manner that can be operationalised by energy utilities.

Power distribution utilities incur significant expenditures due to field operations. While restoration efforts following outages lead to truck rolls in order to identify and rectify faults, there exist several non-outage events that also result in generation of work orders and crew visits. In this paper, we discuss a new problem in the smart grid domain wherein data-driven approaches may be used to improve field operations and customer experience by minimizing the time required to diagnose, locate and rectify non-outage faults. In particular, we discuss how voltage datastream from customer smart meters can be leveraged to pre-inform the crew about such faults and proactively generate work orders. We also conduct a preliminary investigation based on meter data obtained from a large electrical utility in North America.

The increasing penetration of renewable sources like solar energy add new dimensions in planning power grid operations. We study the problem of curtailing a subset of prosumers generating solar power with the twin goals of being close to a target collection and maintaining fairness across prosumers. The problem is complicated by the uncertainty in the amount of energy fed-in by each prosumer and the large problem size in terms of number of prosumers. To meet these challenges, we propose an algorithm based on the Combinatorial Multi-Armed Bandit problem with an approximate Knapsack based oracle. With real-data on solar panel output across multiple prosumers, we are able to demonstrate the effectiveness of the proposed algorithm.

Modern day District Heating and Cooling (DHC) networks are complex interconnections of heat energy sources and heat energy consumers wherein the available energy from the sources is networked to the consumers for meeting their space heating (or cooling) requirements. Often, the energy sources are renewables and thermal run-off from industrial processes which are intermittent. Under extremely exigent conditions (such as very low ambient temperatures), these DHC networks may potentially suffer from energy supply inadequacy. Subsequently, selfish uncoordinated control of heat energy inflow at distributed consumer premises can lead to unfair allocation of the already inadequate energy to different consumers, potentially leading to consumer disgruntlement. Factors such as thermal losses in the network, varying levels of building insulation and different building heat capacities only exacerbate these issues. In this paper, we propose a policy for implementing Demand Response (DR) in DHC networks with an objective of optimizing different fairness based objectives. Specifically, our proposed algorithm estimates dynamically evolving building thermal parameters from continuously recorded temperature sensor measurements from different points in the network. It then uses this knowledge to suggest suitable control of network parameters such as mass flow rate of fluid to the buildings in order to realize the network level thermal fairness based objectives.

HVAC and lighting loads contribute a significant fraction of total energy consumed in office buildings. These loads vary as a function of occupancy and therefore inferring occupancy is vital to optimizing energy efficiency within these buildings. This work presents evaluation and comparison results from a field trial conducted in a large office building, which involved estimating occupancy with the help of existing opportunistic context sources versus instrumented hardware sensors. Our results show that opportunistic sensing yielded an accuracy of 80% in comparison with expensive hardware sensors and may be used to continuously estimate fine-grained workplace occupancy in an inexpensive manner. Moreover the inferred occupancy information may also be used to identify anomalies in thermal management and space utilization within the building.