Conversations around mHealth applications during COVID-19: a network and sentiment analysis of Tweets in Saudi Arabia (Preprint)

BACKGROUND In Saudi Arabia, the first novel coronavirus disease (COVID-19) confirmed case was reported on March 2, 2020, which followed a series of mitigation efforts imposed by the government. The development of specific mobile health applications (mHealth apps) for public use was one of the response strategies employed by the Saudi government. Assessing the impact of these mHealth apps through the opinions of the public posted on social media is crucial to improve mHealth services offered by governments. OBJECTIVE Our aim was to utilize Twitter, as a source of data, to understand conversations and perceptions of users around the use of six mHealth apps developed by the Saudi Ministry of Health, by conducting a network and sentimental analysis of Tweets. The mHealth apps included in our study were “Sehha”, “Mawid”, “Sehhaty”, “Tetamman”, “Tawakkalna”, and “Tabaud”. METHODS We collected mHealth-related Twitter data on December 12, 2020. After including relevant tweets, our final mHealth app networks consisted of a total of 4,995 Twitter users and 8,666 relationships. We used NodeXL to perform the network analysis and visualization. We performed a sentiment analysis using a human-in-the-loop and machine learning approaches. Our manual annotation initially included five classes (positive, neutral, negative, indeterminate, and sarcasm). We excluded indeterminate and sarcasm classes as they usually cause ambiguity for the sentiment classifier. We applied data augmentation techniques to ensure sentiment polarity (positive, negative) in the tweets. The sentiment classifier dataset consisted a total of 4,719 tweets with 26.6% positive, 52.2% neutral, and 21.2% negative. Data preprocessing and normalization were also performed. For building the sentiment classifier, we used the Support Vector Machine with the word2vec embeddings of AraVec. RESULTS Our network analysis showed that “Sehhaty”, “Tawakkalna”, and “Tabaud” had similar patterns and more interactions in conversations than other networks. “Tawakkalna” and “Tabaud” were the largest networks among all, and their conversations were led by various governmental accounts. In comparison, “Sehha”, “Mawid”, “Sehhaty”, and “Tetamman” networks were mainly led by a health sector and media. Our sentiment analysis showed that the majority of Twitter conversations around the six mHealth apps were neutral, which encompassed facts or information pieces, neutral suggestions, and general inquires. Positive tweets focused on appreciation, positive opinions, and expressions around government trust. In contrast, negative tweets included suggestions to overcome weaknesses, issues faced with apps, negative opinions, and negative psychological impact. Our sentiment classifier showed an accuracy, precision, recall, and an F1-score of 85%. CONCLUSIONS Social media can be used as a data source to understand public perceptions on the use of mHealth apps during pandemics. Real-time analytics of social media can help health authorities to address issues and concerns about mHealth apps during public health crises.