Methods of Calculating Usefulness Ratings of Behavioral Facilitation Tweets in Disaster Situations.

When a severe disaster strikes, people transmit various information about the disaster. This kind of information is circulated especially on Twitter, an extremely popular social networking service. Especially, information that encourages people to take some kind of action or not to take any action is called “behavioral facilitation information”, which is known to influence the behaviors of many people. However, the volume of such information is enormous. Information transmitted by numerous unspecified people is not always useful for various victims. As described herein, we propose methods for calculating the usefulness ratings of tweets that include behavioral facilitation information, and which are posted during earthquake disasters. Specifically, we present several methods for calculating the usefulness ratings of tweets as real numbers from 0 to 3 according to a four-step scale (not useful at all, not very useful, somewhat useful, very useful). Then we compare the methods in terms of accuracy. Based on that comparison, the most suitable method for earthquake disasters can be determined. According to an accuracy evaluation experiment using behavioral facilitation tweets actually posted when a certain earthquake occurred, the method using Bidirectional Encoder Representations from Transformers (BERT) achieved the best accuracy (average RMSE or Root-Mean-Square Error calculated using five-fold cross-validation) of 0.39, which is sufficiently high even for unknown data. This result implies that the method can achieve sufficiently high accuracy, even for unknown data.