AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically

Go Generating

AI Traceability

AI parses the academic lineage of this thesis

Generate MRT

AI Insight

AI extracts a summary of this paper

Weibo:

We show that the scoring metrics used in learning unsupervised Bayesian networks do not necessarily optimize the performance of the learned networks in classification

Building classifiers using Bayesian networks

AAAI/IAAI, Vol. 1, pp.1277-1284, (1996)

Cited: 323|Views27

Full Text

Other Links

dl.acm.org dblp.uni-trier.de academic.microsoft.com

Bibtex

Weibo

Keywords

simple bayesian classifierbayesian networku. c. irvine repositoryrecent worknaive bayes classifierMore(8+)

Abstract

Recent work in supervised learning has shown that a surprisingly simple Bayesian classifier with strong assumptions of independence among features, called naive Bayes, is competitive with state of the art classifiers such as C4.5. This fact raises the question of whether a classifier with less restrictive assumptions can perform even bett...More

Code:

Data:

Insight

Full Text

PPT (Upload PPT)

Similar

Reference

Cited

Introduction

A somewhat simplified statement of the problem of supervised learning is as follo¡£w¢ s.
& Thompson 1992)
This classifier learns the conditional probability of each attribute #" given the label !
Classification is done by applying Bayes rule to compute the probability of !
Instantiation of 1 ¤§¥¦¥§¥ ̈¤
This computation is rendered feasible by making a strong independence assumption: all the attributes " are conditionally independent given the value of the label !
The performance of naive Bayes is somewhat surprising given that this is clearly an unrealistic assumption.
It would be erroneous to ignore the correlations between age, education level, and income

Highlights

A somewhat simplified statement of the problem of supervised learning is as follo¡£w¢ s
The first one is the analysis of unsupervised learning of Bayesian networks for classification tasks
We show that the scoring metrics used in learning unsupervised Bayesian networks do not necessarily optimize the performance of the learned networks in classification
Our analysis suggests a possible class of scoring metrics that are suited for this task
The second contribution is the experimental validation of tree augmented naive Bayesian classifiers, Tree Augmented Naive Bayes
In Table 1 we summarize the accuracies of the six learning procedures we discussed in this paper: NBC–the naive Bayesian classifier; Unsup–unsupervised Bayesian networks learned using the minimal description length score; TANÈ —Tree Augmented Naive Bayes networks learned according to Theorem 4.2; Tree Augmented Naive Bayes —smoothed Tree Augmented Naive Bayes networks; C4.5–the decision-tree classifier of (Quinlan 1993); SNBC—the selective naive Bayesian classifier, a wrapper-based feature selection applied to naive Bayes, using the implementation of (John, Kohavi, & Pfleger 1995)

Results

It would be erroneous to ignore the correlations between age, education level, and income.
This fact naturally begs the question of whether the authors can improve the performance of Bayesian classifiers by avoiding unrealistic assumptions about independence

Conclusion

Concluding Remarks

This paper makes two important contributions. The first one is the analysis of unsupervised learning of Bayesian networks for classification tasks.
The second contribution is the experimental validation of tree augmented naive Bayesian classifiers, TAN
This approach was introduced by Geiger (1992), yet was not extensively tested and as a consequence has received little recognition in the machine learning community.
This classification method has attractive computational properties, while at the same time, as the experimental results show, it performs competitively with reported state of the art classifiers

Tables

Table1: Experimental results

Download tables as Excel

Funding

The first author was also supported in part by an IBM Graduate fellowship and NSF Grant IRI-9503109. A Experimental Methodology and Results We run our experiments on the 22 datasets listed in Table 1

Reference

Chow, C. K., and Lui, C. N. 1968. Approximating discrete probability distributions with dependence trees. IEEE Trans. on Info. Theory 14:462–467.
Cormen, T. H.; Leiserson, C. E.; and Rivest, R. L. 1990. Introduction to Algorithms. MIT Press.
Dougherty, J.; Kohavi, R.; and Sahami, M. 1995. Supervised and unsupervised discretization of continuous features. In ML ’95.
Fayyad, U. M., and Irani, K. B. 1993. Multi-interval discretization of continuous-valued attributes for classification learning. In IJCAI ’93, 1022–1027.
Friedman, N., and Goldszmidt, M. 1996. Discretization of continuous attributes while learning Bayesian networks. In ML ’96.
Geiger, D. 1992. An entropy-based learning algorithm of Bayesian conditional trees. In UAI ’92. 92–97.
Heckerman, D.; Geiger, D.; and Chickering, D. M. 1995. Learning Bayesian networks: The combination of knowlege and statistical data. Machine Learning 20:197–243.
Heckerman, D. 1995. A tutorial on learning Bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research.
John, G.; Kohavi, R.; and Pfleger, K. 1995. Irrelevant features and the subset selection problem. In ML ’94. 121–129.
Kohavi, R.; John, G.; Long, R.; Manley, D.; and Pfleger, K. 1994. MLC++: A machine learning library in C++. In Tools with Artificial Intelligence. 740–743.
Kohavi, R. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI ’95. 1137–1143.
Lam, W., and Bacchus, F. 1994. Learning Bayesian belief networks. An approach based on the MDL principle. Computational Intelligence 10:269–293.
Langley, P., and Sage, S. 1994. Induction of selective Bayesian classifiers. In UAI ’94. 399–406.
Langley, P.; Iba, W.; and Thompson, K. 1992. An analysis of bayesian classifiers. In AAAI ’90. 223–228.
Murphy, P. M., and Aha, D. W. 1995. UCI repository of machine learning databases. http://www.ics.uci.edu/̃mlearn/MLRepository.html.
Pazzani, M. J. 1995. Searching for dependencies in Bayesian classifiers. In Proc. of the 5’th Int. Workshop on Artificial Intelligence and Statistics.
Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
Quinlan, J. R. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann.
Singh, M., and Provan, G. M. 1995. A comparison of induction algorithms for selective and non-selective bayesian classifiers. In ML ’95.

Author

Nir Friedman

[0]

Moises Goldszmidt

[0]

Your rating :

No Ratings