Information, Kullback† S. Kullback, S. KullbackSearch for more papers by this author S. Kullback, S. KullbackSearch for more papers by this author First published: 29 September 2014 https://doi.org/10.1002/9781118445112.stat01635Citations: 1 †This article was originally published online in 2006 in Encyclopedia of Statistical Sciences, © John Wiley & Sons, Inc. and republished in Wiley StatsRef: Statistics Reference Online, 2014. Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat No abstract is available for this article.Citing Literature Wiley StatsRef: Statistics Reference OnlineBrowse other articles of this reference work:BROWSE BY TOPICBROWSE A-Z RelatedInformation

The following discussion will form the background for an application of certain of the results of statistical information theory. The results in this section for the particular case of discrete spaces are essentially those for contingency tables in Kullback (1959, Chapter 8). For a similar discussion see Dobrushin (1959), Pinsker (1964). We use the notation in Halmos (1950).

In this chapter we shall consider examples illustrating further applications of the concepts already developed to a number of stochastic processes. In particular, in the examples we shall consider, the use of a linear discriminant functional is of great utility. Before taking up the stochastic processes we believe it desirable to examine first the representation of a random vector and information theoretic techniques as applied in this case. We shall follow procedures which lend themselves to generalization to the case of stochastic processes.

An information‐theoretic statistical procedure for the analysis of categorical or qualitative variables or count data not necessarily arrayed in a multi‐way cross‐classification or contingency table is presented. This procedure can be used to provide a uniform approach to both statistical testing and estimation in various kinds of traffic studies. In particular, a statistical procedure is presented for analyzing counts of violations observed at various intersections during cycles of red left turn arrow or red ball signals. Data was collected at fifteen intersections of different types at different times of the day and located in different geographical locations.

This chapter discusses categorical data problems using information theoretic approach. Concepts of statistical information theory are applied in a very general mathematical formulation to the problems of statistics involving continuous and discrete variables. The impracticability of studying the simultaneous effects of large numbers of variables by contemplation of a multiple cross-classification is now generally recognized. The chapter assumes that the one can have some familiarity with elementary contingency tables and the usual notation, including the dot notation to represent summation over an index. The chapter presents the underlying theory and also illustrates and amplifies the analytic procedures with several examples. In addition, the chapter also focuses on the nonhierarchical models. In recent practice, it is conventional to use hierarchical log linear models. These models are those in which one can include an interaction term and all lower-order interactions involving variables in the higher order term.

A brief review of the minimum discrimination information (MDI) approach in analyzing categorical data is presented in a question -answer format, An example is given to bring out situations in which the MDI approach is more useful. No new results are proved.

The use of the minimum discrimination information (MDI) approach in studying null hypotheses of no interactions on a linear scale in contingency tables is illustrated. Quadratic approximations to the MDI statistic are shown to be related to Waid-type statistics and Neyman's modified chi-square. Follow up analyses when the null hypothesis is not satisfied are illustrated .

Biometrische ZeitschriftVolume 17, Issue 1 p. 1-12 Article Analysis of Interactions Between Categorical Variables S. Kullback, S. Kullback The George Washington Univ. Dept. of Statistics Washington, D. C. 20006, USASearch for more papers by this authorP. N. Reeves, P. N. Reeves The George Washington Univ. Dept. of Statistics Washington, D. C. 20006, USASearch for more papers by this author S. Kullback, S. Kullback The George Washington Univ. Dept. of Statistics Washington, D. C. 20006, USASearch for more papers by this authorP. N. Reeves, P. N. Reeves The George Washington Univ. Dept. of Statistics Washington, D. C. 20006, USASearch for more papers by this author First published: 1975 https://doi.org/10.1002/bimj.19750170102AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinkedInRedditWechat References 1 Dempster, A. P., 1971: An overview of multivariate data analysis. J. of Multivariate Analysis 1, 316. 2 Hammon, G. L., S. E. Jacobs, P. N. Reeves, 1972: Surveys of hospital/medical computing. Hospital Financial Management (Scheduled for September 1972). 3 Johnson, W. D., and G. Koch, 1970: Analysis of qualitative data: linear functions. Health Service Res. 5, 358. 4 Ku, H. H., and S. Kullback, 1968: Interaction in multidimensional contingency tables: An information theoretic approach. Nat. Bur. Stand. J. Res. 72 B, 159. 5 Ku, H. H., R. N. Varner, S. Kullback, 1971: On the analysis of multidimensional contingency tables. J. Am. Stat. Assoc. 66, 55. 6 Kullback, S., 1959, 1968: Information Theory and Statistics. New York, Wiley; Dover Publications Inc., pg. 38. 7 Kullback, S., 1970: Minimum discrimination information estimation and application. Invited paper presented to sixteenth conference on the Design of experiments in army research, development and testing, U.S.A. logistics management center, FT. LEE, VA. 21 October 1970, ARO-D Report 71–3, 1–38. Proceedings of the conference. Volume17, Issue11975Pages 1-12 ReferencesRelatedInformation

Biometrische ZeitschriftVolume 17, Issue 3 p. 139-146 Article On „Multivariate Logit Analysis”︁ Prof. Dr. S. Kullback, Prof. Dr. S. Kullback Department of Statistics The George Washington University Washington, D. C. 2005/2 U.S.A.Search for more papers by this authorM. Fisher, M. Fisher Department of Statistics The George Washington University Washington, D. C. 2005/2 U.S.A.Search for more papers by this author Prof. Dr. S. Kullback, Prof. Dr. S. Kullback Department of Statistics The George Washington University Washington, D. C. 2005/2 U.S.A.Search for more papers by this authorM. Fisher, M. Fisher Department of Statistics The George Washington University Washington, D. C. 2005/2 U.S.A.Search for more papers by this author First published: 1975 https://doi.org/10.1002/bimj.19750170302AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinkedInRedditWechat References Ashford, J. R., and R. D. Sowden, 1970: Multivariate Probit Analysis. Biometrics 26, 535–546. Darroch, J. N., and D. Ratcliff, 1972: Generalised iterative scaling and maximum likelihood. The Annals of Mathematical Statistics 43, 1470–1480. Grizzle, J. E., 1971: Multivariate logit analysis. Biometrics 27, 1057–1062. Grizzle, J. E., C. F. Starmer and G. G. Koch, 1969: Analysis of categorical data by linear models. Biometrics 25, 489–504. Ireland, C. T., and S. Kullback, 1968: Minimum discrimination information estimation. Biometrics 24, 707–713. Ku, H. H., Ruth N. Varner and S. Kullback, 1971: On the analysis of multidimensional contingency tables. Journal American Statistical Association 66, 55–64. Kullback, S., 1959: Information Theory and Statistics. Wiley, New York Information Theory and Statistics. 1968; Dover, New York. Kullback, S., 1973: Estimating and testing interaction parameters in the log-linear model. Biometrische Zeitschrift 15, 371–388. Kullback, S., and Marian Fisher, 1973: Partitioning secondorder interaction in three-way contingency tables. J.R.S.S. Series C (Applied Statistics) 22, 172–184. Volume17, Issue31975Pages 139-146 ReferencesRelatedInformation

(1974). Loglinear Models in Contingency Table Analysis. The American Statistician: Vol. 28, No. 4, pp. 115-122.

The principle of minimum discrimination information estimation and associated techniques are used to consider the partitioning of a three‐way contingency table when second‐order interaction is present. Within the partitioned model a sub‐hypothesis of logit linearity is also studied. Computational procedures utilize a generalized iterative scaling algorithm of Darroch and Ratcliff.