With the explosion of electronic information on web, there is the increasing requirement to obtain the information needed accurately and efficiently. In this article, a method of automatic summarization based on sub topic partition and sentence features is proposed, in which the sentence weight is computed based on LexRank algorithm combining with the score of its own features in every sub topic, such as its length, position, cue words and structure. In addition, we reduce redundancy of candidate sentence collection. With evaluation on six different genres of data sets, our method could get more comprehensive and high-quality summarization with less redundancy than the original LexRank algorithm.

介绍了一种基于统计和未登录词碎片字典的未登录词识别方法.该方法首先在基于分词的基础上对一篇文本进行统计识别未登录词,然后利用未登录词碎片字典识别未登录词.实验结果表明:该方法较已有的未登录词识别方法更适用于识别文本中不断涌现的新词,在未登录词的召回率和准确率性能指标上达到了较好的效果.

The purpose of a focused crawler is to crawl more topical portions of the Internet precisely. How to predict the visit priorities of candidate URLs whose corresponding pages have yet to be fetched is the determining factor in the focused crawler's ability of getting more relevant pages. This paper introduces a comprehensive prediction method to address this problem. In this method, a page partition algorithm that partitions the page into smaller blocks and interclass rules that statistically capture linkage relationships among the topic classes are adopted to help the focused crawler cross tunnel and to enlarge the focused crawler's coverage, URL's address, anchor text and block content are used to predict visit priority more precisely. Experiments are carried out on the target topic of tennis and the results show that crawler based on this method is more effective than a rule-based crawler on harvest ratio.

Similarity calculation of documents is a vital and basic step in text mining, such as document clustering and classification Traditionally, the first step of constructing feature vector of a Chinese document is to segment Chinese text Into words, as there is no apparent word boundary, like space in English The segmentation result weighs heavily on the quality of dictionary A new vector model, FCSNV, is proposed to represent Chinese document without segmentation FCSNV is built in two main steps (1) Use the nodes of Suffix Tree to represent the similarity parts of documents, (2) Use frequent closed nodesets of these nodes to represent dimensions of feature vector Our experiments show that precision is improved effectively when FCSNV is used to measure the most similar documents