Exploring Federated Learning to Trace Depression in Social Media with Language Models

Due to the rising numbers of depression cases in recent years, many initiatives have investigated the use of Machine Learning models to detect depressive symptoms from the individual's presence on social media. To train these models, a dataset is needed. An adequate way of collecting reliable data is to elicit volunteers to agree to share their posts for research. Usually, the volunteer is also requested to answer a depressive inventory to provide the required depression label. However, this data is often sensitive and cannot be shared between research groups, harming reproducibility and collaboration. To address that problem, in this manuscript, we investigate Federated Learning techniques to train a classifier depression method while still preserving the data privacy of individuals. Since social media posts are primarily text-based, we fine-tune language models induced by the Transformer architecture to our task. In our experiments, we simulate the common heterogeneity across clients. Our experiments show that Federated Learning achieves competitive models compared to the centralized version.