Opportunities and challenges of explainable artificial intelligence in medicine

Artificial intelligence (AI) has been successful in a wide variety of domains. However, the opacity of AI makes it hard for users to understand why a certain decision has been reached by the model, which can decrease user trust. Explainable AI, which addresses the implementation of transparency and interpretation of black box models, is especially crucial in the medical domain. In addition to mapping explainability to causability to deliver understandable explanations to users, we argue that the purpose and the presentation style of the explanation should depend on the stakeholder. In this chapter, we discuss the explanatory requirements of physicians, patients, and developers in the clinical flow. We also propose an explainable medical knowledge base diagnosis system to facilitate further development. To protect patient privacy, we present an innovative framework of explainable federated learning. Finally, we briefly mention the limitations and future directions of medical AI.