Introduction
Introduction
We study an interesting phenomenon of social influence locality in a large microblogging network, which suggests that users’ behaviors are mainly influenced by close friends in their ego networks.
1. Definition of influence locality
2. An example
Result
Results
1. Sampling Test for Influence Locality
The above figure shows that for all the time spans, the fraction of active users with 2 active neighbors is about 2 times greater than the fraction of active users with only one active neighbor, i.e. NT=1/NT=0 ~2. Meanwhile, the fraction of active users in the treatment group increases with the number of active neighbors. The test results show strong evidence for the existence of the social influence locality on user’s retweet behaviors. However, we also observe that after 48 hours when the original microblog has been published, the increasing rate slows down with the number of active neighbors, which suggests that the influence decays over time.
In the figure, NT=1 is the average number of active users in the treatment group, and NT=0 is the average number of active users in the control group. We calculate the ratio of the fractions for the two numbers and can conclude that the influence locality exerts positive effect on users’ retweet behaviors if NT=1 /NT=0 > 1.
2. Pairwise Influence
The above figures show that the probability that a user retweets a microblog conditioned on the average random walk probability (a) and sum of the random walk probability (b) of all active neighbors in her ego network. From both figures, we can observe that the random walk based pairwise influence score can be used as a good indicator of the retweet behaviors. we conduct RWR in a user v’s ego network G v and calculate the random walk probability pvi for each active neighbor vi. The random walk probability can be explained as how the influence of an active neighbor can finally reach the given user v via the network connection between them.
3. Structure Influence
We analyze the results by varying the number of active neighbors by 2,3,4,5,6-10, 11-20, and 21-30 respectively. We see that, surprisingly, the retweet probability is negatively correlated with the number of circles.This phenomenon might be explained from the purpose of retweet,people may quickly lose interests to retweet when they find that many of their social circles are already aware of the message. Please note that when calculating the number of circles, we only consider reciprocal (bi-directional) following relationships between users.
2. Retweet prediciton performance of different instantiation functions for influence locality.
Table 2 shows the performance of the comparison methods. The results show that using only the influence locality function to predict retweet behaviors (LRC-Q) can result in a performance comparable with (even better than) that using all the additional features (LRC-B) (+0.6% in terms of F1-measure, -0.3% in terms of accuracy). By combining the influence locality function and the additional features together, we can obtain a bit improvement on performance (+1.65% in terms of F1-measure, +2.49% in terms of accuracy).
We further try different Q = w *g + (1 - w)* f functions for predicting retweet behaviors. Specifically, we first set w = 1 and try seven g functions for pairwise influence and then set w = 0 and try two kinds of f functions for structure influence . The evaluation results are shown in Table 3. We can see that for pairwise influence, g6, which averages the time weighted pairwise influence by using geometric mean, performs the best. The result suggests that the followees with different retweet time actually exert different influence on retweet behaviors. Besides, we also find that arithmetic mean performs poorly comparing with geometric mean for both the time weighted pairwise influence (g5 under-performs g6) and the pairwise influence without time weighting (g2 under-performs g3). This is due to the reason that the estimates of the pairwise influence from the active neighbors are not normally distributed but right-skewed. That is, the majority of pairwise influence are low and a minority of pairwise influence are scattered in a fat right tail. We can see that for structure influence, f2 function considering both the number of active neighbors and the number of circles formed by them performs better than only considering the number of circles, which indicates that there in deed exists strong correlation between the two factors.
Data
Please notice that we put the data set in Dropbox, which has a 200GB per day limitation for the bandwidth. Please try later again if the download is temporally disabled. Thank you!
Good news! You can also dowload the data from baidu cloud and baidu cloud2, where baidu cloud2 is weibo_network.txt.
The data set was crawled in the following ways. To begin with, 100 random users were selected as seed users, and then their followees and followees followees were collected. The crawling process produced in total 1.7 million users and 0.4 billion following relationships among them, with average 200 followees per user. For each user, the crawler collected her 1,000 most recent microblogs (including tweets and retweets). The process resulted in totally 1 billion microblogs. We also crawled all the users profiles which contain name, gender, verification status, #bi-following, #followers, #followees, and #microblogs.
We focus on the retweet behaviors in the microblogging network. Thus we select 300,000 popular microblog diffusion episodes from the data set. Each diffusion episode contains the original microblog and all its retweets. On average each microblog has been retweeted for about 80 times. The sampled data set ensures that for each diffusion episode, the active (retweet) statuses of followees in ones \tau-ego network is completed. Finaly, we obtain the sampled data as follows:
The data set includes six parts:
1. Static Following network
2. Dynamic Following network
3. Retweet behaviors with retweet content
4. Merged retweet behaviors without retweet content
5. User profile
6. Original tweet content
7. User id map
We will introduce each part in detail:
1. Static following network
weibo_network.txt
This file describes the Sina Weibo sub-network at the very first timestamp.
First line consists of two integers, representing the number of users N and number of the follow relationships M respectively.
In the following N lines, each line starts with an integer v1_id, representing the user_id of user v1, followed by another integer k. And the following 2k numbers describes the users "FOLLOWED" by v1, each represented by a user id v2_id and a number indicating the type of relationship. In weibo data set, "1" indicates that user v1 and v2 have a reciprocal FOLLOW relationship, while "0" indicates that their relationship is not reciprocal.
"weibo_network_final_sample.txt" provides the first 10,001 lines of "weibo_network.txt".
Note that user_id has been hashed! Refer to uidlist.txt for the original uid in Sina.
2. Dynamic following network
graph_170w_1month.txt
This file describes the Sina Weibo sub-network at every timestamp ( one timestamp is one day) in one month from 2012.9.28 to 2012.10.29. The format is as follows:
"id1 id2 t"
means person id1 follows person id2 at timestamp t
Sample line:
"0 11 1"
3. Retweet Behaviors with retweet content
===============================================
There are many repost*.txt files. In each repost*.txt, the format is as follows:
"
original_mid original_uid original_time retweet_num
N
retweet_uid retweet_time retweet_mid
retweet_text
"
Sample:
"
3509462670824667 1618051664 2012-11-06-20:22:42 9571
2
120567 2012-11-06-21:39:49 3509482074105278
some chinese characters
593399 2012-11-07-21:49:26 3509846865102966
//@some chinese characters
"
Each original tweet contains 2+2N lines.
First line consists the original tweet id, the original user id, the original publishing time, and the actual number of retweets
Second line is the number of retweets in our dataset.
In the following 2N lines, each first line consisits the retweet user id, the retweet time, and the retweet id.
and each second line is the text content of the retweet.
Please note that the retweets for the same original tweet may be contained in different repost*.txt, you should merge them by yourself.
4. Merge retweet behaviors without retweet content
===============================================
total.txt, the format is as follows:
"
original_mid original_time original_uid retweet_num
retweet_uid retweet_time retweet_uid retweet_time ....
"
Sample:
"
3479380645050771 2012-08-15-20:07:32 1500872 3
1500872 2012-08-15-20:41:02 1500872 2012-08-15-20:41:20
"
Each original tweet contains 2 lines.
First line consists the original tweet id, the original publishing time, the original user id, and the actual number of retweets
Second line contains all the retweets with each one contains the retweet user id and the retweet time.
Second format
repost_data.txt
This file consists of reposting logs of 300,000 posts. The reposting logs of each post are represented by several lines. In the first line, two integers A and B, separated by a [TAB], describe the post_id and the number of reposts respectively. In the following B lines, each line describes an action of reposting by two integers T and U, represent the time stamp when this post was reposted, and the user_id who reposted this post repectively. Time stamp T is encoded by how many seconds have passed since 1970-01-01-00:00:00.
---------------------------------
Sample:
1000 341
[post_id] [number_of_reposts]
1324266090 1569353
[time_stamp] [user_id]
1324266137 591987
[time_stamp] [user_id]
...
---------------------------------
Note that the number of reposts is not actually how many times the post has been reposted in the entire Weibo network, but how many unique users in our user set have reposted this post. Please also note that if a user reposted the same post for multiple times, I only kept the first reposting log and removed the others. Thus each user will only appear at most once in a reposting logs of a post.
5. User profile
=======================================
user_profile1.txt and user_profile2.txt, the format is as follows:
"
user id
the number of the reciprocal relationships
city
verification status
the number of followers
location
province
the number of followees
name
gender
the time when created
verified_type
the number of published/retweeted tweets
description
"
Sample:
"
3
7
1000
False
100
some chinese characters
11
80
c
f
2009-10-29-20:36:25
-1
1
...
"
6. Original tweet content
===================================
root_content.txt, the format is as follows:
"
original tweet id
content
"
sample:
"
3515638699605834
some chinese characters
"
Each original tweet contains two lines with the first line represents the tweet id and the second line represents the content.
7. User id map
=========================
uidlist.txt
The number k on the ith (start from 0) line represents mapping the original user id k to the new id i.
8. Tweet and Retweets with indexed content.
We also process the tweet and retweet content to transfer the chineses characters into index number, in case that the researchers who do not understand Chinese would like to utilize the dataset.
Three files are included:
1) Original tweet content
===================================
Root_Content.txt, the format is as follows:
"
original tweet id
content
[@]
[link]
"
Each original tweet contains two to four lines with the first line representing the tweet id and the second line representing the content.
The original content has been processed and the meaningful words have been hashed.
The third line represents the @ list in the original content, if there is no Id being at, there is no such line or with only a '@' followed by no id
The fourth line represents the link in the original content, if there is no link, there is no such line
sample:
"
3515748850831323
8757292 9446082 21012919 22107098 20926192 845774 5628107 9369656 14735005 6214073 4523200 12130042 4144274 16646575 21367256 11379159 1404772 19538310 11316649 14645874 1404772 5288691 20714768 14516287 14645874 4315741 2965587 19515364 11379159 5912581 17680225 14645874 15980511 10055284 1404772 8738613 21188182 16159013 12035269 9446082 14707579 3306679 13261282 14735005 18696133 14714428 8493523 9657711 240811 14645874 751792 15863260
@ yakumocri weibo_account1 weibo_account2 weibo_account3
link http://t.cn/zj4yJim
"
2) Number of retweets to the original tweet
===================================
Weibo_Retweet_Num.txt, the format is as follows:
"
original tweet id number of retweets
"
Each original tweet contains one line,the first number stands for the tweet id and the second represents the number of retweets to this tweet in our dataset.
3) Retweet content
===================================
Retweet_Content.txt, the format is as follows:
"
original_mid original_uid original_time retweet_num
N
retweet_uid retweet_time retweet_mid
retweet_content
[@]
[retweet]
[link]
"
First line contains the original tweet id, the original user id, the original publishing time, and the actual number of retweets
Second line is the number of retweets in our dataset.
In the following lines, two to five lines describe the information of one retweet, where each first line consisits of the retweet user id, the retweet time, and the retweet id, each second line is the text content of the retweet which has been hashed. If there are no meaningful words, there is simply nothing in this line, but the line exsits.
Third line represents the @ list in the retweet content, if there is no Id being at, there is no such line
Fourth line represents the retweet list in the whole content, if retweeting directly from the original, not from other people. there is no such line.
Fifth line represents the link in the whole content, if there is no link, there is no such line.
sample:
"
3493810665875895 2401987563 2012-09-24-15:47:16 6
4
1768721815 2012-09-24-21:36:51 3493898641466073
3913348 7579362 16603489 18546250 8493523 7579362 11379159 12681902 4333484 18506807 13734979
retweet weibo_account1
1877247847 2012-09-25-07:07:56 3494042354282019
4582554 21155697 7318323 20455830 10407442
retweet weibo_account2
1574777217 2012-09-25-11:29:40 3494108221769908
3550998 16362438 9552703 21786125
@ pynix
retweet weibo_account3 weibo_account4
1574777217 2012-09-24-21:33:57 3493897910926942
7562241 12681902 14338803 94162 15382724 21188182 11737162 7579362 589407 3219525
"
4) Chinese word map
=======================
WordTable.txt records the word index in the whole dataset, the format is as follows:
First line indicates the number of the whold words.
Second line is "a b c", which means the ath word c occurs b times in the whole dataset.
Thanks Biao Liu, Yang Liu, XiaoXiao Li, Honglei Zhuang and Zhanpeng Fang for data process.
Code
Please find all the codes [here].
Reference
Jing Zhang, Biao Liu, Jie Tang, Ting Chen, and Juanzi Li. Social Influence Locality for Modeling Retweeting Behaviors. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI'13). pp. 2761-2767. [PDF] [Data&Code] [Poster]
Jing Zhang, Jie Tang, Juanzi Li, Yang Liu, and Chunxiao Xing. 2015. Who Influenced You? Predicting Retweet via Social Influence Locality. ACM Trans. Knowl. Discov. Data 9, 3, Article 25 (April 2015), 26 pages.