该文主要解决中文搜索引擎的查询纠错问题.错误的查询,已经偏离用户真实的搜索意图时,搜索质量很差,甚至导致搜索结果数为零.为此该文提出了一种服务于实际搜索引擎,较为完整的查询纠错方案.该文重点描述了纠错查询候选生成、纠错查询候选评价、以及基于核函数,挑选最优纠错查询候选等内容.通过在开放测试集上的准确率/召回率验证,以及在搜索引擎中实际的DCG评测,该文的方案都取得了较好的效果.

网页搜索中的查询时效性是指查询对新闻网页的需求.这种时间相关的因素,在网页排序过程中用于平衡其他非时间性因素,使排序更好地满足用户体验.为此该文提出了一种查询时效性的实时计算模型:从用户搜索和媒体报道两个角度,分别对时效性建模,然后这两种不同来源的时效性相互补充,综合计算某个时刻用户搜索某个查询时,其综合时效性得分.这个量化得分在网页排序阶段用于提高或抑制新闻网页的露出;同时也为网页搜索结果中展现新闻直达区提供依据.在人工评测以及用户点击通过率统计上,该模型均取得了不错的实际效果.

Bilingual translation pairs play an import role in many NLP applications,such as cross language information retrieval and machine translation.The translation of proper names,out of vocabulary words,idioms and technical terminologies is one of the key factors that affect the performance of the systems.However,these translations can hardly be found in the traditional bilingual dictionary.This paper proposes a new method to automatically extract high quality translation pairs from Wikipedia based on the wide area coverage and data structure,the method not only can learn common patterns,but also learn many patterns that can hardly be found by human beings.The method contains three steps:1)extract translation pairs from the language toolbox of the Wikipedia.They can be heuristic for the next step;2)learn patterns of translation pairs with the knowledge of PAT-Array gained from the previous work;3)extract other translation pairs automatically using the learned patterns.Our experimental results show the accuracy can reach 90.4%.

冗长查询指用户提交的句子成份复杂的查询。当前的搜索引擎对于关键字的检索取得了较好的结果。但是对于冗长的查询,如果将所有词作为关键字进行检索,往往只能返回相当有限的结果。我们尝试利用关键词之间的词语关联度,发现语义蕴含,删除"信息量"小的关键词,提高检索的效果。对于实验结果,我们分别从"面向机器"和"面向用户"两个角度进行评价。在"面向机器"的评价部分,我们根据搜索引擎返回结果的标红率和结果数进行自动评价;在"面向用户"的评价部分,我们对搜索结果文档进行人工评价。实验结果表明,我们的方法能够明显提高检索结果的数量和质量。

Markov Logic Network has the ability to handling the complex representation and the uncertainty of first-order logic and probabilistic graphical models.An entity resolution method based on Markov logic network and property extraction algorithm employing ontology and web search is proposed to improve the performance of named entity resolution for unstructured data based on Markov logic network.The method is then applied to the resolution of Chinese geographic names.The experimental result shows that the proposed method is effective in geographic entity resolution.

Sentiment classification is used to identify whether the opinion expressed in a document is positive or negative. In this paper, we present an approach to do documentary-level sentiment classification by modeling description of topical terms. The motivation of this work stems from the observation that the global document classification will benefit greatly by examining the way of a topical term to give opinion in its local sentence context. Two sentence-level sentiment description models, namely positive and negative Topical Term Description Models, are constructed for each topical term. When analyzing a document, the Topical Term Description Models generate divergence to support the classification of its sentiment at the sentence-level which in turn can be used to decide the whole document classification collectively. The results of the experiments prove that our proposed method is effective. It is also shown that our results are comparable to the state-of-art results on a publicly available movie review corpus and a Chinese digital product review corpus. This is quite encouraging to us and motivates us to have further investigation on the development of a more effective topical term related description model in the future.

Both a general domain-independent bottom-up multi-level model and an algorithm for establishing the taxonomic relation of Chinese ontology are proposed. The model consists of extracting domain vocabularies and establishing taxonomic relation, with the consideration of characteristics unique to Chinese natural language. By establishing the semantic forests of domain vocabularies and then using the existing semantic dictionary or machine-readable dictionary (MRD), the proposed algorithm can integrate these semantic forests together to establish the taxonomic relation. Experimental results show that the proposed algorithm is feasible and effective in establishing the integrated taxonomic relation among domain vocabularies and concepts.

For the multiword expression (MWE) extraction, the multiple sequence alignment (MSA) is proposed on the motivation of gene recognition. Because textual sequence is similar to gene sequence in pattern analysis. This MSA technique is combined with error-driven rules, with the improved efficiency beyond the traditional methods. It provides a guarantee for the MWE recall. It uses the dynamic programming method to prevent candidates from combinational explosion, and provides a global solution for pattern extraction instead of sub-pattern redundancy. Consequently, it has accurate measures for flexible patterns. In experiment, some advanced statistical measures are performed for ranking candidates. In the comparison experiment, the MSA approach achieved better results.

In this paper, we develop previous AVS models and propose a conceptual model to represent adjectival modification in Chinese, with the emphasis laid on its polysemy. Through a method based on rough pattern, we can extract automatically this model from the corresponding senses of adjectival entries in a machine-readable dictionary. This work belongs to a family of research into the automatic compiling of CFL pedagogical e-dictionary based on Modern Chinese Standard Dictionary.

Sentiment classification is used to identify whether the opinion expressed in a document is positive or negative. In this paper, we present an evaluation modeling approach to document-level sentiment classification. The motivation of this work stems from the observation that the global document classification can benefit greatly by learning how a topical term is evaluated in its local sentence context. Two sentence-level sentiment evaluation models, namely positive and negative models, are constructed for each topical term. When analyzing a document, the evaluation models generate divergence to support sentence classification that in turn can be used to decide on the whole document classification collectively. When evaluated on a public available movie review corpus, the experimental results are comparable with the ones published. This is quite encouraging to us and motivates us to further investigate how to develop more effective evaluation models in the future.

This paper presents the participation of the Hong Kong Polytechnic University in the TAC 2008 competition. The systems for the participated tracks are introduced respectively.

Many natural language processing tasks, such as text classification, text clustering, text summarization, and information retrieval etc., cannot miss the step-relevance measure of inter-sentence. However, many of the current NLP system always calculate not the inter-sentence relevance but their similarity. In fact, similarity means differently from relevance. The similarity measure can be acquired by comparing the exterior tokens of inter-sentences, but relevance measure can be obtained only by comparing the interior meaning of the sentences. In this paper, we described a method to explore the quantified conceptual relations of word-pairs by using the definition of a lexical item in modern Chinese standard dictionary, and proposed a practical approach to measure the inter-sentence relevance. The results of the examples show that our approach can solve the problem of how to measure the relevance of two sentences without (or very low) similarity but with a certain relevance. This method is also compatible with the current cosine similarity method.

In Modern Chinese Standard dictionary, adjectival modification is represented as a set of synonyms or antonyms which are tagged by attributes. Through a method based on rough pattern, we can extract automatically these widely-accepted attributes from its machine readable dictionary. This work provides a new method to construct a common attribute database for NLP in Chinese.

The dependence analysis is usually the key for improving the performance of text retrieval. Compared with the statistical value of a conceptual relationship, the recognition of relation type between concepts is more meaningful. In this paper, we explored a bootstrapping method for automatically extracting semantic patterns from a large-scale corpus to identify the geographical "is part of" relationship between Chinese location concepts. The experiments showed that the pattern set generated by our method achieves higher coverage and precision than DIPRE does.

The dependence analysis is usually the key for improving the performance of text retrieval. Compared with the statistical value of a conceptual relationship, the recognition of relation type between concepts is more meaningful. In this paper, we explored a bootstrapping method for automatically extracting semantic patterns from a large-scale corpus to identify the geographical "be part of" relationship between Chinese location concepts in contexts. Our contributions different from other bootstrapping methods lie in: (1) introducing a bi-sequence alignment algorithm in bio-informatics to generating candidate patterns, and (2) giving a new evaluating metric for patterns' confidence to enhance their extracting qualities in next iteration. In terms of automatic recognition of "be part of" relationship, the experiments showed that the pattern set generated by our method achieves higher coverage and precision than DIPRE does.

This paper presents an approach to utilize a Chinese machine-readable dictionary to acquire a conceptual network with five semantic relations; and then, the application of this conceptual resource is discussed text retrieval. Because of the low accuracy of current Chinese parser, our method is different from other traditional dictionary processing that need parsing the explanations of words when constructing this noun-based conceptual network. This construction is designed to be a new three-step process by statistical learning mechanism. This method acquires a relatively satisfying conceptual network based on Chinese nouns. In its application, this network is available for providing linkage-based knowledge for text retrieval.

In this paper, we exploit a Chinese machine-readable dictionary to extract the conceptual knowledge, i.e. the pairs involving in hypernym, (artificiality) material, (artificiality) function and (medicine) usage from the corresponding definitions of nominal entries. Our method focuses on (1) constructing the extraction patterns and (2) the statistical decision for applying these patterns. Therefore our work is designed to be a new three-step procedure. Firstly, annotate the definitions of a number of nominal entries that are used as training samples of these four attributes and contextual linguistic features; secondly, design different patterns for extracting such conceptual knowledge, and learn the applicability of the patterns by a Maximum Entropy (ME) classifier to decide whether a pattern can be used in current context or not; at last, apply these patterns to the remaining nominal entries of the dictionary, and we achieve relatively satisfying results.

提出了一种基于语言建模的文本情感分类的方法.将文本的情感倾向标记为"赞扬"或"批评",可以为文本提供主题之外的语义信息.为此提出了从训练数据中分别估计出代表"赞扬"和"批评"两种情感倾向的语言模型,然后通过比较测试文本自身的语言模型和这两种训练好的情感模型之间的Kullback-Leibler距离,分类测试文本的思路.各个模型的参数分别选用词形特征的unigram和bigram,而相应的参数估计也分别尝试了最大似然和平滑两种策略.当在电影评论语料上和代表不同分类模型的支持向量机及朴素贝叶斯分类器进行比较时,语言建模的方法表现出了较好的分类性能和鲁棒性.

This paper presents a language modeling approach to the sentiment detection problem. It captures the subtle information in text processing to character the semantic orientation of documents as "thumb up" (positive) or "thumb down" (negative). To handle this problem, we propose an idea to estimate both the positive and negative language models from training collections. Tests are done through computing the Kullback-Leibler divergence between the language model estimated from test document and these two trained sentiment models. We assert the polarity of a test document by observing whether its language model is close to the trained "thumb up" model or the "thumb down" model. When compared with an outstanding classifier, i.e., SVMs on movie review corpus, language modeling approach showed its better performance.

Text clustering is widely used to partition texts into clusters so that intra-cluster texts are similar and inter-cluster ones are dissimilar. In this paper we present a new similarity between texts on the basis of combining cosine measure with the quantified conceptual relations via interpolation for text clustering. These relations derive from the entries and the words in their definitions in a traditional dictionary, which are quantified under the assumption that the entries and their definitions have the equivalent meaning, respectively. This kind of relations is considered as hidden knowledge for text clustering. Under the framework of Partitioning Around Medoids algorithm, the new interpolated similarity improves the performance of clustering system significantly in terms of optimizing four hard and soft criterion functions. Our results show that introducing conceptual knowledge from the unstructured dictionary tends to provide contributions for text clusterings.