AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
We proposed a novel algorithm to compute these causal Shapley values, based on causal chain graphs
Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models
NIPS 2020, (2020)
Shapley values underlie one of the most popular model-agnostic methods within explainable artificial intelligence. These values are designed to attribute the difference between a model's prediction and an average baseline to the different features used as input to the model. Being based on solid game-theoretic principles, Shapley values...More
PPT (Upload PPT)
- Complex machine learning models like deep neural networks and ensemble methods like random forest and gradient boosting machines may well outperform simpler approaches such as linear regression or single decision trees, but are noticeably harder to interpret.
- In this paper the authors show that there is no need to resort to asymmetric Shapley values to incorporate causal knowledge: applying conditioning by intervention instead of conditioning by observation is sufficient.
- Complex machine learning models like deep neural networks and ensemble methods like random forest and gradient boosting machines may well outperform simpler approaches such as linear regression or single decision trees, but are noticeably harder to interpret
- (3) Making use of causal chain graphs , we propose a practical approach for computing causal Shapley values and illustrate this on a real-world example
- In this paper we show that there is no need to resort to asymmetric Shapley values to incorporate causal knowledge: applying conditioning by intervention instead of conditioning by observation is sufficient
- This paper introduced causal Shapley values, a model-agnostic approach to split a model’s prediction of the target variable for an individual data point into contributions of the features that are used as input to the model, where each contribution aims to estimate the total effect of that feature on the target and can be decomposed into a direct and an indirect effect
- We proposed a novel algorithm to compute these causal Shapley values, based on causal chain graphs
- All that a practitioner needs to provide is a partial causal order and a way to interpret dependencies between features that are on an equal footing
- Marginal Shapley values end up with the same explanation for the predicted bike rental on both days, ignoring that the temperature in winter is higher than normal for the time of year and in fall lower.
- In the case of a chain, asymmetric and symmetric causal Shapley values provide different explanations.
- With a uniform distribution over all features and no further assumption w.r.t. the causal ordering of X1 and X2, the Shapley values are φ1 = φ2 = 1/4 when the prediction f equals 1, and φ1 = φ2 = −1/4 for f = 0: completely symmetric.
- The authors' approach is inspired by , but extends it in various aspects: it provides a formalization in terms of causal chain graphs, applies to both symmetric and asymmetric Shapley values, and correctly distinguishes between dependencies that are due to confounding and mutual interactions.
- To illustrate the difference between marginal and causal Shapley values, the authors consider the bike rental dataset from , where the authors take as features the number of days since January 2011, two cyclical variables to represent season, the temperature, feeling temperature, wind speed, and humidity.
- The difference between asymmetric, causal, and marginal Shapley values clearly shows when the authors consider two days, October 10 and December 3, 2012, with more or less the same temperature of 13 and 13.27 degrees Celsius, and predicted bike counts of 6117 and 6241, respectively.
- The marginal Shapley values provide more or less the same explanation for both days, essentially only considering the more direct effect temp.
- The causal Shapley values nicely balance the two extremes, giving credit to both season and temperature, to provide a sensible, but still different explanation for the two days.
- This paper introduced causal Shapley values, a model-agnostic approach to split a model’s prediction of the target variable for an individual data point into contributions of the features that are used as input to the model, where each contribution aims to estimate the total effect of that feature on the target and can be decomposed into a direct and an indirect effect.
- Additional user studies should confirm to what extent explanations provided by causal Shapley values align with the needs and requirements of practitioners in real-world settings.
- Kjersti Aas, Martin Jullum, and Anders Løland. Explaining individual predictions when features are dependent: More accurate approximations to Shapley values. arXiv preprint arXiv:1903.10464, 2019.
- Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José MF Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 648–657, 2020.
- Anupam Datta, Shayak Sen, and Yair Zick. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE Symposium on Security and Privacy (SP), pages 598–617. IEEE, 2016.
- Lilian Edwards and Michael Veale. Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for. Duke Law & Technology Review, 16:18, 2017.
- Hadi Fanaee-T and Joao Gama. Event labeling combining ensemble detectors and background knowledge. Progress in Artificial Intelligence, 2:113–127, 2014.
- Christopher Frye, Ilya Feige, and Colin Rowat. Asymmetric Shapley values: Incorporating causal knowledge into model-agnostic explainability. arXiv preprint arXiv:1910.06358, 2019.
- Tobias Gerstenberg, Noah Goodman, David Lagnado, and Joshua Tenenbaum. Noisy Newtons: Unifying process and dependency accounts of causal attribution. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 34, pages 378–383, 2012.
- Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. Feature relevance quantification in explainable AI: A causal problem. In International Conference on Artificial Intelligence and Statistics, pages 2907–2916. PMLR, 2020.
- Daniel Kahneman and Dale T Miller. Norm theory: Comparing reality to its alternatives. Psychological Review, 93(2):136, 1986.
- Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. Interpreting interpretability: Understanding data scientists’ use of interpretability yools for machine learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–14, 2020.
- I Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle Friedler. Problems with Shapley-value-based explanations as feature importance measures. arXiv preprint arXiv:2002.11097, 2020.
- Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems, pages 4066–4076, 2017.
- Steffen L Lauritzen and Thomas S Richardson. Chain graph models and their causal interpretations. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(3):321–348, 2002.
- David Lewis. Causation. The Journal of Philosophy, 70(17):556–567, 1974.
- Stan Lipovetsky and Michael Conklin. Analysis of regression in game theory approach. Applied Stochastic Models in Business and Industry, 17(4):319–330, 2001.
- Tania Lombrozo and Nadya Vasilyeva. Causal explanation. Oxford Handbook of Causal Reasoning, pages 415–432, 2017.
- Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1):2522– 5839, 2020.
- Scott M Lundberg, Gabriel G Erion, and Su-In Lee. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888, 2018.
- Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pages 4765–4774, 2017.
- Luke Merrick and Ankur Taly. The explanation game: Explaining machine learning models with cooperative game theory. arXiv preprint arXiv:1909.08128, 2019.
- Brent Mittelstadt, Chris Russell, and Sandra Wachter. Explaining explanations in AI. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 279–288, 2019.
- Chong Weng Ong, Denise M O’Driscoll, Helen Truby, Matthew T Naughton, and Garun S Hamilton. The reciprocal interaction between obesity and obstructive sleep apnoea. Sleep Medicine Reviews, 17(2):123–131, 2013.
- Judea Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669–688, 1995.
- Judea Pearl. The do-calculus revisited. arXiv preprint arXiv:1210.4852, 2012.
- Bob Rehder. A causal-model theory of conceptual representation and categorization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(6):1141, 2003.
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144, 2016.
- Lloyd S Shapley. A value for n-person games. Contributions to the Theory of Games, 2(28):307– 317, 1953.
- Steven Sloman. Causal models: How people think about the world and its alternatives. Oxford University Press, 2005.
- Elliott Sober. Apportioning causal responsibility. The Journal of Philosophy, 85(6):303–318, 1988.
- Barbara A Spellman. Crediting causality. Journal of Experimental Psychology: General, 126(4):323, 1997.
- Erik Štrumbelj and Igor Kononenko. Explaining prediction models and individual predictions with feature contributions. Knowledge and Information Systems, 41(3):647–665, 2014.
- European Union. EU General Data Protection Regulation (GDPR): Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC (General Data Protection Regulation), OJ 2016 L 119/1, 2016.
- Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law and Technology, 31:841, 2017.