AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We proposed a novel algorithm to compute these causal Shapley values, based on causal chain graphs

Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models

NIPS 2020, (2020)

Cited by: 0|Views18
EI
Full Text
Bibtex
Weibo

Abstract

Shapley values underlie one of the most popular model-agnostic methods within explainable artificial intelligence. These values are designed to attribute the difference between a model's prediction and an average baseline to the different features used as input to the model. Being based on solid game-theoretic principles, Shapley values...More

Code:

Data:

0
Introduction
  • Complex machine learning models like deep neural networks and ensemble methods like random forest and gradient boosting machines may well outperform simpler approaches such as linear regression or single decision trees, but are noticeably harder to interpret.
  • In this paper the authors show that there is no need to resort to asymmetric Shapley values to incorporate causal knowledge: applying conditioning by intervention instead of conditioning by observation is sufficient.
Highlights
  • Complex machine learning models like deep neural networks and ensemble methods like random forest and gradient boosting machines may well outperform simpler approaches such as linear regression or single decision trees, but are noticeably harder to interpret
  • (3) Making use of causal chain graphs [13], we propose a practical approach for computing causal Shapley values and illustrate this on a real-world example
  • In this paper we show that there is no need to resort to asymmetric Shapley values to incorporate causal knowledge: applying conditioning by intervention instead of conditioning by observation is sufficient
  • This paper introduced causal Shapley values, a model-agnostic approach to split a model’s prediction of the target variable for an individual data point into contributions of the features that are used as input to the model, where each contribution aims to estimate the total effect of that feature on the target and can be decomposed into a direct and an indirect effect
  • We proposed a novel algorithm to compute these causal Shapley values, based on causal chain graphs
  • All that a practitioner needs to provide is a partial causal order and a way to interpret dependencies between features that are on an equal footing
Results
  • Marginal Shapley values end up with the same explanation for the predicted bike rental on both days, ignoring that the temperature in winter is higher than normal for the time of year and in fall lower.
  • In the case of a chain, asymmetric and symmetric causal Shapley values provide different explanations.
  • With a uniform distribution over all features and no further assumption w.r.t. the causal ordering of X1 and X2, the Shapley values are φ1 = φ2 = 1/4 when the prediction f equals 1, and φ1 = φ2 = −1/4 for f = 0: completely symmetric.
  • The authors' approach is inspired by [6], but extends it in various aspects: it provides a formalization in terms of causal chain graphs, applies to both symmetric and asymmetric Shapley values, and correctly distinguishes between dependencies that are due to confounding and mutual interactions.
  • To illustrate the difference between marginal and causal Shapley values, the authors consider the bike rental dataset from [5], where the authors take as features the number of days since January 2011, two cyclical variables to represent season, the temperature, feeling temperature, wind speed, and humidity.
  • The difference between asymmetric, causal, and marginal Shapley values clearly shows when the authors consider two days, October 10 and December 3, 2012, with more or less the same temperature of 13 and 13.27 degrees Celsius, and predicted bike counts of 6117 and 6241, respectively.
  • The marginal Shapley values provide more or less the same explanation for both days, essentially only considering the more direct effect temp.
  • The causal Shapley values nicely balance the two extremes, giving credit to both season and temperature, to provide a sensible, but still different explanation for the two days.
Conclusion
  • This paper introduced causal Shapley values, a model-agnostic approach to split a model’s prediction of the target variable for an individual data point into contributions of the features that are used as input to the model, where each contribution aims to estimate the total effect of that feature on the target and can be decomposed into a direct and an indirect effect.
  • Additional user studies should confirm to what extent explanations provided by causal Shapley values align with the needs and requirements of practitioners in real-world settings.
Reference
  • Kjersti Aas, Martin Jullum, and Anders Løland. Explaining individual predictions when features are dependent: More accurate approximations to Shapley values. arXiv preprint arXiv:1903.10464, 2019.
    Findings
  • Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José MF Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 648–657, 2020.
    Google ScholarLocate open access versionFindings
  • Anupam Datta, Shayak Sen, and Yair Zick. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE Symposium on Security and Privacy (SP), pages 598–617. IEEE, 2016.
    Google ScholarLocate open access versionFindings
  • Lilian Edwards and Michael Veale. Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for. Duke Law & Technology Review, 16:18, 2017.
    Google ScholarLocate open access versionFindings
  • Hadi Fanaee-T and Joao Gama. Event labeling combining ensemble detectors and background knowledge. Progress in Artificial Intelligence, 2:113–127, 2014.
    Google ScholarLocate open access versionFindings
  • Christopher Frye, Ilya Feige, and Colin Rowat. Asymmetric Shapley values: Incorporating causal knowledge into model-agnostic explainability. arXiv preprint arXiv:1910.06358, 2019.
    Findings
  • Tobias Gerstenberg, Noah Goodman, David Lagnado, and Joshua Tenenbaum. Noisy Newtons: Unifying process and dependency accounts of causal attribution. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 34, pages 378–383, 2012.
    Google ScholarLocate open access versionFindings
  • Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. Feature relevance quantification in explainable AI: A causal problem. In International Conference on Artificial Intelligence and Statistics, pages 2907–2916. PMLR, 2020.
    Google ScholarLocate open access versionFindings
  • Daniel Kahneman and Dale T Miller. Norm theory: Comparing reality to its alternatives. Psychological Review, 93(2):136, 1986.
    Google ScholarLocate open access versionFindings
  • Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. Interpreting interpretability: Understanding data scientists’ use of interpretability yools for machine learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–14, 2020.
    Google ScholarLocate open access versionFindings
  • I Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle Friedler. Problems with Shapley-value-based explanations as feature importance measures. arXiv preprint arXiv:2002.11097, 2020.
    Findings
  • Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In Advances in Neural Information Processing Systems, pages 4066–4076, 2017.
    Google ScholarLocate open access versionFindings
  • Steffen L Lauritzen and Thomas S Richardson. Chain graph models and their causal interpretations. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(3):321–348, 2002.
    Google ScholarLocate open access versionFindings
  • David Lewis. Causation. The Journal of Philosophy, 70(17):556–567, 1974.
    Google ScholarLocate open access versionFindings
  • Stan Lipovetsky and Michael Conklin. Analysis of regression in game theory approach. Applied Stochastic Models in Business and Industry, 17(4):319–330, 2001.
    Google ScholarLocate open access versionFindings
  • Tania Lombrozo and Nadya Vasilyeva. Causal explanation. Oxford Handbook of Causal Reasoning, pages 415–432, 2017.
    Google ScholarLocate open access versionFindings
  • Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1):2522– 5839, 2020.
    Google ScholarLocate open access versionFindings
  • Scott M Lundberg, Gabriel G Erion, and Su-In Lee. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888, 2018.
    Findings
  • Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pages 4765–4774, 2017.
    Google ScholarLocate open access versionFindings
  • Luke Merrick and Ankur Taly. The explanation game: Explaining machine learning models with cooperative game theory. arXiv preprint arXiv:1909.08128, 2019.
    Findings
  • Brent Mittelstadt, Chris Russell, and Sandra Wachter. Explaining explanations in AI. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 279–288, 2019.
    Google ScholarLocate open access versionFindings
  • Chong Weng Ong, Denise M O’Driscoll, Helen Truby, Matthew T Naughton, and Garun S Hamilton. The reciprocal interaction between obesity and obstructive sleep apnoea. Sleep Medicine Reviews, 17(2):123–131, 2013.
    Google ScholarLocate open access versionFindings
  • Judea Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669–688, 1995.
    Google ScholarLocate open access versionFindings
  • Judea Pearl. The do-calculus revisited. arXiv preprint arXiv:1210.4852, 2012.
    Findings
  • Bob Rehder. A causal-model theory of conceptual representation and categorization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(6):1141, 2003.
    Google ScholarLocate open access versionFindings
  • Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144, 2016.
    Google ScholarLocate open access versionFindings
  • Lloyd S Shapley. A value for n-person games. Contributions to the Theory of Games, 2(28):307– 317, 1953.
    Google ScholarLocate open access versionFindings
  • Steven Sloman. Causal models: How people think about the world and its alternatives. Oxford University Press, 2005.
    Google ScholarFindings
  • Elliott Sober. Apportioning causal responsibility. The Journal of Philosophy, 85(6):303–318, 1988.
    Google ScholarLocate open access versionFindings
  • Barbara A Spellman. Crediting causality. Journal of Experimental Psychology: General, 126(4):323, 1997.
    Google ScholarLocate open access versionFindings
  • Erik Štrumbelj and Igor Kononenko. Explaining prediction models and individual predictions with feature contributions. Knowledge and Information Systems, 41(3):647–665, 2014.
    Google ScholarLocate open access versionFindings
  • European Union. EU General Data Protection Regulation (GDPR): Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC (General Data Protection Regulation), OJ 2016 L 119/1, 2016.
    Google ScholarLocate open access versionFindings
  • Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law and Technology, 31:841, 2017.
    Google ScholarLocate open access versionFindings
Author
Evi Sijben
Evi Sijben
Ioan Gabriel Bucur
Ioan Gabriel Bucur
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科