Interpretable Machine Learning Methods for in Vitro Pharmaceutical Formulation Development

Background: Machine learning has become an alternative approach for pharmaceutical formulation development. However, many machine learning applications in pharmaceutics only focus on model performance rather than model interpretability. Aim: This study aims to propose an attention-based deep neural network (DNN) for pharmaceutical formulation development. Methods: An attention-based DNN, AttPharm, was proposed. AttPharm separately handled feature values and feature physical meaning by representation learning to successfully apply the attention mechanism to the pharmaceutical tabular data. Furthermore, the distributions of the attention weights were computed using AttPharm. Two post hoc methods, local interpretable model-agnostic explanation (LIME) and Tree-SHAP, were utilized to obtain the post hoc model interpretability for lightGBM. Results: The results demonstrated that AttPharm significantly improved the model performance of plain neural networks on a pharmaceutical cyclodextrin dataset because the attention mechanism could extract related features and find minute variation. Notably, the attention weights were analyzed, which illustrated global and local feature-level and sample-level model interpretability, thus providing insights for formulation design. Comparing with post hoc methods, AttPharm can be used without the concern of the faithfulness of interpretability. Conclusion: This is the first step in applying the attention-based DNN to pharmaceutical formulation development. Considering the importance of model interpretability, the proposed approach may have a wide range of applications in pharmaceutics.