AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We show that the scoring metrics used in learning unsupervised Bayesian networks do not necessarily optimize the performance of the learned networks in classification

Building classifiers using Bayesian networks

AAAI/IAAI, Vol. 1, pp.1277-1284, (1996)

Cited: 323|Views27
EI
Full Text
Bibtex
Weibo

Abstract

Recent work in supervised learning has shown that a surprisingly simple Bayesian classifier with strong assumptions of independence among features, called naive Bayes, is competitive with state of the art classifiers such as C4.5. This fact raises the question of whether a classifier with less restrictive assumptions can perform even bett...More

Code:

Data:

Introduction
  • A somewhat simplified statement of the problem of supervised learning is as follo¡£w¢ s.
  • & Thompson 1992)
  • This classifier learns the conditional probability of each attribute #" given the label !
  • Classification is done by applying Bayes rule to compute the probability of !
  • Instantiation of 1 ¤§¥¦¥§¥ ̈¤
  • This computation is rendered feasible by making a strong independence assumption: all the attributes " are conditionally independent given the value of the label !
  • The performance of naive Bayes is somewhat surprising given that this is clearly an unrealistic assumption.
  • It would be erroneous to ignore the correlations between age, education level, and income
Highlights
  • A somewhat simplified statement of the problem of supervised learning is as follo¡£w¢ s
  • The first one is the analysis of unsupervised learning of Bayesian networks for classification tasks
  • We show that the scoring metrics used in learning unsupervised Bayesian networks do not necessarily optimize the performance of the learned networks in classification
  • Our analysis suggests a possible class of scoring metrics that are suited for this task
  • The second contribution is the experimental validation of tree augmented naive Bayesian classifiers, Tree Augmented Naive Bayes
  • In Table 1 we summarize the accuracies of the six learning procedures we discussed in this paper: NBC–the naive Bayesian classifier; Unsup–unsupervised Bayesian networks learned using the minimal description length score; TANÈ —Tree Augmented Naive Bayes networks learned according to Theorem 4.2; Tree Augmented Naive Bayes —smoothed Tree Augmented Naive Bayes networks; C4.5–the decision-tree classifier of (Quinlan 1993); SNBC—the selective naive Bayesian classifier, a wrapper-based feature selection applied to naive Bayes, using the implementation of (John, Kohavi, & Pfleger 1995)
Results
  • It would be erroneous to ignore the correlations between age, education level, and income.
  • This fact naturally begs the question of whether the authors can improve the performance of Bayesian classifiers by avoiding unrealistic assumptions about independence
Conclusion
  • Concluding Remarks

    This paper makes two important contributions. The first one is the analysis of unsupervised learning of Bayesian networks for classification tasks.
  • The second contribution is the experimental validation of tree augmented naive Bayesian classifiers, TAN
  • This approach was introduced by Geiger (1992), yet was not extensively tested and as a consequence has received little recognition in the machine learning community.
  • This classification method has attractive computational properties, while at the same time, as the experimental results show, it performs competitively with reported state of the art classifiers
Tables
  • Table1: Experimental results
Download tables as Excel
Funding
  • The first author was also supported in part by an IBM Graduate fellowship and NSF Grant IRI-9503109. A Experimental Methodology and Results We run our experiments on the 22 datasets listed in Table 1
Reference
  • Chow, C. K., and Lui, C. N. 1968. Approximating discrete probability distributions with dependence trees. IEEE Trans. on Info. Theory 14:462–467.
    Google ScholarLocate open access versionFindings
  • Cormen, T. H.; Leiserson, C. E.; and Rivest, R. L. 1990. Introduction to Algorithms. MIT Press.
    Google ScholarFindings
  • Dougherty, J.; Kohavi, R.; and Sahami, M. 1995. Supervised and unsupervised discretization of continuous features. In ML ’95.
    Google ScholarFindings
  • Fayyad, U. M., and Irani, K. B. 1993. Multi-interval discretization of continuous-valued attributes for classification learning. In IJCAI ’93, 1022–1027.
    Google ScholarLocate open access versionFindings
  • Friedman, N., and Goldszmidt, M. 1996. Discretization of continuous attributes while learning Bayesian networks. In ML ’96.
    Google ScholarFindings
  • Geiger, D. 1992. An entropy-based learning algorithm of Bayesian conditional trees. In UAI ’92. 92–97.
    Google ScholarLocate open access versionFindings
  • Heckerman, D.; Geiger, D.; and Chickering, D. M. 1995. Learning Bayesian networks: The combination of knowlege and statistical data. Machine Learning 20:197–243.
    Google ScholarLocate open access versionFindings
  • Heckerman, D. 1995. A tutorial on learning Bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research.
    Google ScholarFindings
  • John, G.; Kohavi, R.; and Pfleger, K. 1995. Irrelevant features and the subset selection problem. In ML ’94. 121–129.
    Google ScholarLocate open access versionFindings
  • Kohavi, R.; John, G.; Long, R.; Manley, D.; and Pfleger, K. 1994. MLC++: A machine learning library in C++. In Tools with Artificial Intelligence. 740–743.
    Google ScholarFindings
  • Kohavi, R. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI ’95. 1137–1143.
    Google ScholarLocate open access versionFindings
  • Lam, W., and Bacchus, F. 1994. Learning Bayesian belief networks. An approach based on the MDL principle. Computational Intelligence 10:269–293.
    Google ScholarLocate open access versionFindings
  • Langley, P., and Sage, S. 1994. Induction of selective Bayesian classifiers. In UAI ’94. 399–406.
    Google ScholarLocate open access versionFindings
  • Langley, P.; Iba, W.; and Thompson, K. 1992. An analysis of bayesian classifiers. In AAAI ’90. 223–228.
    Google ScholarLocate open access versionFindings
  • Murphy, P. M., and Aha, D. W. 1995. UCI repository of machine learning databases. http://www.ics.uci.edu/̃mlearn/MLRepository.html.
    Findings
  • Pazzani, M. J. 1995. Searching for dependencies in Bayesian classifiers. In Proc. of the 5’th Int. Workshop on Artificial Intelligence and Statistics.
    Google ScholarLocate open access versionFindings
  • Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
    Google ScholarFindings
  • Quinlan, J. R. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann.
    Google ScholarFindings
  • Singh, M., and Provan, G. M. 1995. A comparison of induction algorithms for selective and non-selective bayesian classifiers. In ML ’95.
    Google ScholarFindings
0
Your rating :

No Ratings

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn