Detecting Community Kernels in Large Social Networks

In many social networks, there exist two types of users that exhibit different influence and different behavior. For instance, statistics have shown that less than 1% of the Twitter users (e.g. entertainers, politicians, writers) produce 50% of its content [1], while the others (e.g. fans, followers, readers) have much less influence and completely different social behavior. In this paper, we define and explore a novel problem called community kernel detection in order to uncover the hidden community structure in large social networks. We discover that influential users pay closer attention to those who are more similar to them, which leads to a natural partition into different community kernels.

This work has been published in ICDM 2013. If you use data sets provided or refer to the model, please cite:

@inproceedings{Wang:11ICDM,

   author = {Liaoruo Wang and Tiancheng Lou and Jie Tang and John Hopcroft},

   title = {Detecting Community Kernels in Large Social Networks},

   booktitle = {ICDM'11},

   year = {2011},

   pages = {784-793},

}

The full paper can be downloaded here[pdf].

Kernel Community

The Pareto principle (a.k.a. 80-20 rule) [3] exists almost everywhere. For example, 80% of a country’s land is owned by 20% of the population, and 80% of a company’s sales revenue comes from 20% of its clients. This is also the case for many social networks. In these networks, there exist two types of users that exhibit different influence and different behavior. For instance, it has been shown that less than 1% of the Twitter users (e.g. entertainers, politicians, writers) produce roughly 50% of the content on the micro-blogging site [1], while the other 99% (e.g. fans, followers, readers) have much less influence and completely different social behavior. Then, an interesting question is: “how do these influential users interact with each other?” Further, influential users are typically followed more than others. For example, Oprah Winfrey has more than 5 million followers, and Barack Obama has more than 7 million. Hence, another interesting question is: “what is the underlying structure between influential users and their followers?”.

To clearly demonstrate this, we present an example from the Twitter network as shown in Fig. 1. The left figure is an input of the Twitter following network with three entertainers (Oprah Winfrey, Ashton Kutcher, and Demi Moore) and two politicians (Barack Obama and Al Gore) as well as some of their followers. This input represents a typical network structure with a few influential users connected with the rest of the network via a large number of links. To detect the community structure of this network, we consider Newman’s algorithm [2], a state-of-the-art method based on modularity. The middle figure shows the community structure obtained by Newman’s algorithm. We observe that, since there are a large number of connections between each influential user and its followers, Newman’s algorithm tends to partition the influential users into different communities and to group them with their respective followers. The lack of ability to distinguish influential users from their numerous followers is a key problem with this method. The right figure shows the community structure obtained by our algorithm WEBA later introduced in Section III. By contrast, this is exactly what one would expect a community detection algorithm to discover: two community kernels (one of entertainers and one of politicians) consist of influential users and two auxiliary communities associated with the kernels. Thus, in this paper, we refer to this problem as community kernel detection, which includes two parts: (1) how to distinguish influential users (kernel members) from others, and (2) how to detect the community structure (community kernels) among influential users and their respective auxiliary communities.

Fig. 1. An illustration of community kernel detection on the Twitter network. The left figure shows the original Twitter network (three entertainers and two politicians with their followers), the middle figure shows the five communities detected by Newman’s algorithm [2], and the right figure shows two community kernels and their corresponding auxiliary communities detected by our algorithm WEBA.

[Download]

We propose GREEDY and WEBA, two efficient algorithms for finding community kernels in large social networks. GREEDY is based on maximum cardinality search, while WEBA formalizes the problem in an optimization framework. We conduct experiments on three large social networks: Twitter, Wikipedia, and Coauthor, which show that WEBA achieves an average 15%– 50% performance improvement over the other state-of-the-art algorithms, and WEBA is on average 6–2,000 times faster in detecting community kernels. Fig. 2 shows an efficiency comparison of eight algorithms on the three networks.

Fig. 2. Efficiency comparison of WEBA and GREEDY with comparative algorithms (no parallelization).

All codes used in this paper can be downloaded here, which includes: [Download]

• Local Spectral Partitioning (LSP) [8]: community detection algorithm based on conductance. This algorithm is in general a spectral-based graph partitioning method.
• d-LSP: apply LSP to high-degree (top 20%) nodes to find communities. Degree is considered as the relative influence of each vertex.
• p-LSP: apply LSP to high-PageRank (top 20%) nodes [17] to find communities. PageRank is considered as the relative influence of each vertex.
• METIS+MQI [5][6]: community detection algorithm based on conductance. This algorithm is a flow-based partitioning method for finding low-conductance cuts.
• LOUVAIN [16]: community detection algorithm based on modularity. This algorithm is in general a greedy optimization method.
• NEWMAN1 [4]: community detection algorithm based on betweenness. This algorithm is in general an agglomerative hierarchical clustering method.
• NEWMAN2 [2]: community detection algorithm based on modularity. This algorithm interprets community detection as a spectral problem in linear algebra.
• alpha-beta [11]: community detection algorithm based on (alpha, beta)-community.

The first seven algorithms are based on the assumption that communities are densely connected internally and sparsely connected externally, while the last algorithm -, similar to WEBA, allows communities to have dense external connections. In addition, d-LSP and p-LSP consider the relative influence of vertices, while the other five do not. All algorithms are implemented using C++.

[Download]

Our experiments are conducted on three different real-world social networks:

• Coauthor (a co-authorship network with 822,415 nodes and 2,928,360 undirected edges). Each vertex represents an author and each edge represents a co-author relation.
• Wikipedia (a co-editorship network with 310,990 nodes and 10,780,996 undirected edges crawled from wikipedia. org). Each vertex represents a Wikipedia editor and each edge represents a co-editing relation.
• Twitter (a following network with 465,023 nodes and 833,590 directed edges crawled from twitter.com). Each vertex represents a Twitter user account and each edge represents a following relation. It is well-known that the web displays a bow-tie structure [20], where 30% of the vertices are strongly connected. We conduct a bow-tie analysis on the Twitter network, and discover that only 8% (38,913) of the vertices are strongly connected.

WEBA and GREEDY perform much better than the other comparative algorithms for detecting community kernels. On average, WEBA achieves a 14–50% and a 15–42% performance improvement over comparative algorithms in terms of F1-score for the wikipedia and coauthor networks. GREEDY also achieves a better performance than comparative algorithms, but on average works 10% and 7% less well than WEBA.

Liaoruo Wang, Tiancheng Lou, Jie Tang, and John E. Hopcroft. Detecting Community Kernels in Large Social Networks. In Proceedings of 2011 IEEE International Conference on Data Mining (ICDM'11). pp. 784-793. [PDF] [Slides]