This paper covers the implementation and testing of a Deep Belief Network (DBN) for the purposes of document classification. Document classification is important because of the increasing need for document organization, particularly journal articles and online information. Many popular uses of document classification are for email spam filtering, topic-specific search engine (where we only want to search documents that fall under a certain topic), or sentiment detection (to determine whether a document has more of a positive or negative tone) [1]. DBNs using a mostly unsupervised clustering algorithm for training are implemented. The clustering algorithm groups documents together that fall under the same category. Once the documents are grouped, a supervised learning algorithm (SVM in this case) is used to “assign” labels to the groups. More information and a background on DBNs is given in the following section. Testing of the DBN is primarily performed along three axes: 1) the number of hidden neurons in each layer of the DBN (which determines a “shape” for the overall network), 2) the number of layers in the DBN, and 3) the number of iterations that are used to train each layer of the DBN. Additionally, classification results of the DBN are compared with results obtained from using support vector machine (SVM) and Naïve Bayes (NB) classifiers. Some final experiments investigate how vocabulary size and word preprocessing affects performance. Preliminary results indicate that the DBN, given the implementations used in this paper, is not a viable alternative to other forms of document classifiers. The DBN has a significantly longer training time, and no matter how many training iterations are used, the DBN appears to have worse accuracy than either the SVM or NB classifiers. A number of suggestions for improvement are given in the final section, at least one of which should greatly increase the performance of the DBN.
分享
引用
