## AI helps you reading Science

## AI Insight

AI extracts a summary of this paper

Weibo:

# Building classifiers using Bayesian networks

AAAI/IAAI, Vol. 1, pp.1277-1284, (1996)

EI

Keywords

Abstract

Recent work in supervised learning has shown that a surprisingly simple Bayesian classifier with strong assumptions of independence among features, called naive Bayes, is competitive with state of the art classifiers such as C4.5. This fact raises the question of whether a classifier with less restrictive assumptions can perform even bett...More

Code:

Data:

Introduction

- A somewhat simplified statement of the problem of supervised learning is as follo¡£w¢ s.
- & Thompson 1992)
- This classifier learns the conditional probability of each attribute #" given the label !
- Classification is done by applying Bayes rule to compute the probability of !
- Instantiation of 1 ¤§¥¦¥§¥ ̈¤
- This computation is rendered feasible by making a strong independence assumption: all the attributes " are conditionally independent given the value of the label !
- The performance of naive Bayes is somewhat surprising given that this is clearly an unrealistic assumption.
- It would be erroneous to ignore the correlations between age, education level, and income

Highlights

- A somewhat simplified statement of the problem of supervised learning is as follo¡£w¢ s
- The first one is the analysis of unsupervised learning of Bayesian networks for classification tasks
- We show that the scoring metrics used in learning unsupervised Bayesian networks do not necessarily optimize the performance of the learned networks in classification
- Our analysis suggests a possible class of scoring metrics that are suited for this task
- The second contribution is the experimental validation of tree augmented naive Bayesian classifiers, Tree Augmented Naive Bayes
- In Table 1 we summarize the accuracies of the six learning procedures we discussed in this paper: NBC–the naive Bayesian classifier; Unsup–unsupervised Bayesian networks learned using the minimal description length score; TANÈ —Tree Augmented Naive Bayes networks learned according to Theorem 4.2; Tree Augmented Naive Bayes —smoothed Tree Augmented Naive Bayes networks; C4.5–the decision-tree classifier of (Quinlan 1993); SNBC—the selective naive Bayesian classifier, a wrapper-based feature selection applied to naive Bayes, using the implementation of (John, Kohavi, & Pfleger 1995)

Results

- It would be erroneous to ignore the correlations between age, education level, and income.
- This fact naturally begs the question of whether the authors can improve the performance of Bayesian classifiers by avoiding unrealistic assumptions about independence

Conclusion

**Concluding Remarks**

This paper makes two important contributions. The first one is the analysis of unsupervised learning of Bayesian networks for classification tasks.- The second contribution is the experimental validation of tree augmented naive Bayesian classifiers, TAN
- This approach was introduced by Geiger (1992), yet was not extensively tested and as a consequence has received little recognition in the machine learning community.
- This classification method has attractive computational properties, while at the same time, as the experimental results show, it performs competitively with reported state of the art classifiers

- Table1: Experimental results

Funding

- The first author was also supported in part by an IBM Graduate fellowship and NSF Grant IRI-9503109. A Experimental Methodology and Results We run our experiments on the 22 datasets listed in Table 1

Reference

- Chow, C. K., and Lui, C. N. 1968. Approximating discrete probability distributions with dependence trees. IEEE Trans. on Info. Theory 14:462–467.
- Cormen, T. H.; Leiserson, C. E.; and Rivest, R. L. 1990. Introduction to Algorithms. MIT Press.
- Dougherty, J.; Kohavi, R.; and Sahami, M. 1995. Supervised and unsupervised discretization of continuous features. In ML ’95.
- Fayyad, U. M., and Irani, K. B. 1993. Multi-interval discretization of continuous-valued attributes for classification learning. In IJCAI ’93, 1022–1027.
- Friedman, N., and Goldszmidt, M. 1996. Discretization of continuous attributes while learning Bayesian networks. In ML ’96.
- Geiger, D. 1992. An entropy-based learning algorithm of Bayesian conditional trees. In UAI ’92. 92–97.
- Heckerman, D.; Geiger, D.; and Chickering, D. M. 1995. Learning Bayesian networks: The combination of knowlege and statistical data. Machine Learning 20:197–243.
- Heckerman, D. 1995. A tutorial on learning Bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research.
- John, G.; Kohavi, R.; and Pfleger, K. 1995. Irrelevant features and the subset selection problem. In ML ’94. 121–129.
- Kohavi, R.; John, G.; Long, R.; Manley, D.; and Pfleger, K. 1994. MLC++: A machine learning library in C++. In Tools with Artificial Intelligence. 740–743.
- Kohavi, R. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI ’95. 1137–1143.
- Lam, W., and Bacchus, F. 1994. Learning Bayesian belief networks. An approach based on the MDL principle. Computational Intelligence 10:269–293.
- Langley, P., and Sage, S. 1994. Induction of selective Bayesian classifiers. In UAI ’94. 399–406.
- Langley, P.; Iba, W.; and Thompson, K. 1992. An analysis of bayesian classifiers. In AAAI ’90. 223–228.
- Murphy, P. M., and Aha, D. W. 1995. UCI repository of machine learning databases. http://www.ics.uci.edu/̃mlearn/MLRepository.html.
- Pazzani, M. J. 1995. Searching for dependencies in Bayesian classifiers. In Proc. of the 5’th Int. Workshop on Artificial Intelligence and Statistics.
- Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
- Quinlan, J. R. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann.
- Singh, M., and Provan, G. M. 1995. A comparison of induction algorithms for selective and non-selective bayesian classifiers. In ML ’95.

Tags

Comments

数据免责声明

页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果，我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问，可以通过电子邮件方式联系我们：report@aminer.cn