Methodologies for Monitoring Mental Health on Twitter: Systematic Review

Background : The use of social media data in predicting mental health outcomes has the potential to allow for continuous monitoring of mental health and well-being, and to provide timely information that can supplement traditional clinical assessments. However, it is crucial that the methodologies used to create models for this purpose are of high quality from both a mental health and a machine learning perspective. Twitter has been a popular choice of social media due to the accessibility of its data, but access to big datasets is not a guarantee of robust results.Objective : To review the current methodologies used in the literature for predicting mental health outcomes from Twitter data, with a focus on the quality of underlying mental health data and machine learning methods used.Methods : A systematic search was used across six databases with keywords relating to mental health disorders, algorithms, and social media. 2,759 records were screened, from which 165 papers were analysed. Information about methodologies for data acquisition, pre-processing, model creation and validation were collected, as well as replicability and ethical considerations.Results : The 165 papers reviewed used 120 primary datasets. There were an additional 8 datasets identified that were not described in enough detail to include, and 10 papers did not describe their datasets at all. Of these 120 datasets, only 16 had access to ground truth data (i.e. known characteristics) about the mental health disorders of social media users. The other 104 datasets collected data by searching key words or phrases, which may not be representative of patterns of Twitter use for those with mental health disorders. The annotation of mental health disorders for classification labels was variable and 68 out of 120 datasets had no ground truth or clinical input on this annotation. Despite being a common mental health disorder, anxiety received little attention.Conclusions : The sharing of high-quality ground truth datasets is crucial for the development of trustworthy algorithms which have clinical and research utility. Further collaboration across disciplines and contexts is encouraged to better understand what types of predictions will be useful in supporting management and identification of mental health disorders. A series of recommendations for researchers in this field and for the wider research community are made with the aim of enhancing the quality and utility of future outputs.