Using Machine Learning To Understand Suicide: A New Approach To Classifying Australian Coroner’s Court Decisions

Abstract We aimed to demonstrate how a large collection of publicly accessible Australian Coroner’s Court case files (n=4459) (2009-2019) can be automatically classified for determination of death by suicide, presence of mental health disorder and sex of deceased via Natural Language Processing (NLP) methods - supervised machine learning and unsupervised dictionary-based and string search based approaches. We achieved superior levels of accuracy in the machine learning classification (Gradient Boosting vs. Random Forest baseline) of deaths by suicide of 83.3% (sensitivity = 85.1%, Specificity = 79.1%) and an accuracy of 98.3% for the dictionary-based classification of mental health disorder, as defined by the OCD-10 (sensitivity = 99.0%, specificity = 97.9%). Our machine learning approach automatically classified 24.2% (1078/4459) of the case files as referring to deaths by suicide while 63.7% (2940/4459) where classified as exhibiting a mental health disorder1. We employed a two-stage machine learning approach involving feature engineering, followed by predictive modelling in the second. Feature engineering involved several steps including removal of low value text, parts of speech analysis, term document weighting and topic clustering. Predictive classification involved extensive hyperparameter tuning to yield the most accurate model. We validated our models against a manually pre-coded subsample of case files, and also via binary logistic regression to test the contribution of each classified mental health disorder against determinations of deaths by suicide according to extant literature. This validation step confirmed elevated odds of suicide attributed to diagnoses of Depression, Schizophrenia and Obsessive Compulsive Disorder. Finally, we offer a short case study to demonstrate the efficacy of our approach in investigating a subset of case findings referring to suicides resulting from family violence. We offer a proof of concept model that demonstrates an objective and scalable approach to the analysis of legal texts. The use of NLP methods in analysing Coroner's Court case findings has important implications for the ongoing development of a real-time surveillance of suicide system in Australia.