Self-interpretable Bayesian Rule Lists Model with Deep Prototype Layer

Bayesian rule lists (BRL) model aims to provide interpretability for prediction and decision-making process, where researchers have conducted sufficient work to make such probabilistic model applicable to be integrated into more complex architecture. Deep neural net with prototype layers initially created to deal with dimension reduction, visualization and image data interpretability in computer vision tasks, is now also widely used in other fields due to its pluggable interpretability. However, pure BRL is lack of model expressiveness especially for settings where approximation for highly complex non-linear mapping is requisite. Meanwhile, the prototype deep net, although can output certain kind of prototypes, relies on other interpretable models or problem definition to provide concrete interpretable logics for the generated prototypes. In this paper, we mainly deal with difficulty of providing both interpretability and decent prediction precision for decision process, proposing a deep neural network with prototype layers based on BRL model, known as self-interpretable Bayesian rule lists (SiBRL). This model provides interpretability through Bayesian rules and prototype with decent model performance based on expressive deep net. We first use truncated Poisson distribution and uniform distribution to generate rules and decision points which are put into deep net together with training data, and then compute the likelihood based on deep net architecture with Bayesian rule process. Finally, cutting plane algorithm (CPA) is used to optimize the parameters and hyperparameters in the BRL model with the objective being the likelihood given by the deep net part. Experiment based on a default prediction dataset shows that our model can provide reasonable interpretability while keeping prediction precision superior to non-ensembled machine learning models such as decision tree and BRL.