Web User Profiling

Introduction

Web user profiling, also referred to as user profile extraction and user profile mining, has long been viewed as an important and challenging problem in Web mining and natural language processing. Related studies can be traced back to 30 years ago. The general task of web user profiling is to extract ``semantic''-based user profile attributes (e.g., contact information, educational history, experience, and biography) from the unstructured Web.

Web user profiling can be applied in many applications, and is becoming necessary in most social-related systems. With a large and high-quality profile database, we could easily identify what kind of information we should recommend to a specific user. In e-commerce, one could also leverage the profile information to locate targeted customers for a new product.

In this paper, we try to revisit this problem from the perspective of big data. Specifically, to avoid error propagation, we propose a unified approach framework to process all the extraction subtasks together in one step. Moreover, rather than figuring out new ways of reducing noise (or redundancy), we develop a simple but very effective approach for extracting user profile attributes using the redundant information in the big Web data. To incorporate redundant information to improve the extraction accuracy, we propose a markov logic factor graph (MagicFG) model to formalize human knowledge as first-order logics in the model.

*Publication*

Xiaotao Gu, Hong Yang, Jie Tang* and Jing Zhang. The 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM'16). August, San Francisco, 2016 (Best Student Paper Runner-Up)
[pdf] [slides] [code&data]

OR Bibtex

@inproceedings{Gu:16ASONAM,
   author = {Xiaotao Gu, Hong Yang, Jie Tang and Jing Zhang},
   title = {Web User Profiling using Data Redundancy},
   booktitle = {Proceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2016},
   series = {ASONAM '16},
   year = {2016},
location = {San Francisco, USA},
   publisher = {ACM}
}

Motivation

Despite much research conducted in this field to automate the process of profile extraction, the problem remains largely unsolved. To generate the profile for a specific user, a usual approach is to first find relevant Web pages of this user from the Web, and then use machine learning models (e.g., CRFs) to extract profile attributes from the pages. State-of-the-art accuracy achieved in the two stages are around 90.0%, respectively. For example, F1 score is reported as 92% for the task of homepage finding conducted by Tang et al. and when 87% for extracting profile attributes from the homepage. From independent views, the performance sounds good. However, taking error propagation into consideration, the overall accuracy of the approach that combines the two stages together drops to 80%. More seriously, with the rapid growth of the data volume on the Web, the problem becomes even more challenging, as the big Web data contains much more noisy and redundant data.

All these problems motivated us to propose a unified approach framework to process all the extraction subtasks together in one step. Moreover, rather than figuring out new ways of reducing noise (or redundancy), we develop a simple but very effective approach for extracting user profile attributes using the redundant information in the big Web data. To incorporate redundant information to improve the extraction accuracy, we propose a markov logic factor graph (MagicFG) model to formalize human knowledge as first-order logics in the model.

We compare our approach with several state-of-the-art methods for profiling. As shown in the following figure, our approach significantly outperforms existing methods for the task of Email extraction and Gender inference in terms of F1-score.

Data

Credit to the team leaded by Professor Jibing Gong and Haopeng Zhang from YSU (Yanshan University) for labeling some of the data.

1. Email [download]

For Email extraction, we labeled a dataset of around 2000 people, for training and testing. The name list is selected randomly from aminer.org. For each person in this name list, we leveraged Google to search for and extract candidate email addresses. We used contact information in the Aminer system as most of the ground truths, and had some human experts (without knowledge about our classification model) to label and double-check the data.

*Data Format*

The data is a json file, encoded from a dictionary in the format of :

{
PersonName: {
"email_list": [
valid emails
],
"wrong_email": [

invalid emails
]
}

}

where "email list" is the set of valid emails of this person from the extracted candidates, while "wrong emails" is the set of invalid ones.

2. Gender [download]

For Gender inference, we offer a labeled xlsx file of around 2400 people from the Aminer system, with fields including name, organization, position and homepage.

Code

[download] [example]

Welcome to use the code under the terms of the license for research or non commercial purposes, however please acknowledge its use with a citation:

@inproceedings{Gu:16ASONAM,
   author = {Xiaotao Gu, Hong Yang, Jie Tang and Jing Zhang},
   title = {Web User Profiling using Data Redundancy},
   booktitle = {Proceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2016},
   series = {ASONAM '16},
   year = {2016},
location = {San Francisco, USA},
   publisher = {ACM}
}

1. implementation

Markov Logic Factor Graph Model (MagicFG)

Credit to Zhanpeng Fang for helping tune this code.

Usage: ./OpenCRF -est -niter $numIter -gradientstep $gradientStep -trainfile $trainFile -dstmodel $modelFile

Options:

task: -est

-niter int : number of iterations

-nbpiter int : number of iterations in belif propgation

-srcmodel string : (for -estc) the model to load

-dstmodel string : model file to save

-method string : methods (lbfgs/gradient), default: gradient

-gradientstep double : learning rate

-hasvalue : [default] attributes with values (format: attr:val)

-novalue : attributes without values (format: attr)

-trainfile string : train file

-testfile string : test file

2. data preparation

First n line describes n variable nodes and their attribute factors in the factor graph mode.

*attributes*

A line starts with character '?' or '+', where '?' indicates the label of this node is UNKNOWN, while '+' implies a known label. The following character is the label of this node. Note that even if we assume the label is unknown for the model, you can still give the label here, if it is actually available. The model will ignore the label of '?' sample during the training process, but it can be used in the evaluation. Separated by space, the rest tokens in a line are organized as "[feature name]:[feature value]", depicting the attrubutes of the node. The feature value can be a real number, and the feature name should be a string without space.

-----------

example:

+0 prefix_contain_last_name:1 prefix_contain_first_name:1

**This line depicts a node with a known label '0'. Four features are given: "prefix_contain_last_name" and "prefix_contain_first_name". The values of both features are 1, while the other features appearing in other lines but not this line, has the value 0.

-----------

*edge*

The following m lines establish the factors between nodes. The factors can only be established in two nodes. A line started by "#edge", followed by two positive integers i and j, indicates a factor between node i and node j. (Node described in the i-th line of this file is node i) At the end of this line is a string, indicating the name of this factor. Factors with the same name share the same form of function g(yi, yj).

-----------

example:

#edge 143 4289 same-domain

**Nodes in the 143-rd line and the 4289-th line are correlated with a "same-domain" factor.

-----------

*logic*

For this problem, we simply express Markov Logic as preferred node values for edges of each specific kind. At the bottom of the feature file, we have several lines starting with "#logic", followed by the name of the edge (relationship), and the preferred values of the two nodes of the edge (ordered). If an edge of this kind doesn't have nodes with this preferred values, the possibility of its existence will decline.

-----------

example:

#logic same_domain_with_invalid 1 0

**A pair of nodes with a relationship as "a has the same domain with an invalid instance b" has the preferred value "a is correct while b is incorrect".

-----------

3. output

The accuracy, precision, recall, and F1-score are calculated in each iteration, over both the sets of All nodes and Unlabeled nodes. Note that the result is only valid for binary classification. MagicFG will take the label of the first line as "positive" label.

"pred.txt" gives the prediction result of all the nodes (including known nodes). The result is presented by the label id. 0 represents the first kind of label detected in the train file, 1 represents the second kind of label encountered in the file, 2 for the third kind of label, etc.

"marginal.txt" gives the marginal probability of each label for every node. Each line, the marginal probability is presented in the order of label id (p(y=label0), p(y=labe1), p(y=label2)....)