The phenomenon was observed that more “not applicable” (NA) responses occurred for the items near the end of a survey instrument than for those at the beginning. In this article, the authors present evidence that this phenomenon represents missing data in disguise in a survey study, especially when the survey is long. They explore the implications for survey data analysis in this situation. Practical recommendations are made for survey instrument construction and survey data analysis to minimize the potential adverse effect of such missing data in disguise. Limitations of the study are also discussed.

The performances of predictive discriminant analysis (PDA) and logistic regression (LR) for the 2-group classification problem were compared. The authors used a fully crossed 3-factor experimental design (sample size, group proportions, and equal or unequal covariance matrices) and 2 data patterns. When the 2 groups had equal covariance matrices, PDA and LR performed comparably for the conditions of both equal and unequal group proportions. When the 2 groups had unequal covariance matrices (4:1, as implemented in this study) and very different group proportions, PDA and LR differed somewhat with regard to the classification error rates of the 2 groups, but the classification error rates of the 2 methods for the total sample remained comparable. Sample size played a relatively minor role in the classification accuracy of the 2 methods, except when LR was used under relatively small sample-size conditions.

A Monte Carlo simulation study was conducted to investigate the effects on structural equation modeling (SEM) fit indexes of sample size, estimation method, and model specification. Based on a balanced experimental design, samples were generated from a prespecified population covariance matrix and fitted to structural equation models with different degrees of model misspecification. Ten SEM fit indexes were studied. Two primary conclusions were suggested: (a) some fit indexes appear to be noncomparable in terms of the information they provide about model fit for misspecified models and (b) estimation method strongly influenced almost all the fit indexes examined, especially for misspecified models. These 2 issues do not seem to have drawn enough attention from SEM practitioners. Future research should study not only different models vis‐à‐vis model complexity, but a wider range of model specification conditions, including correctly specified models and models specified incorrectly to varying degrees.

Standard statistical methods are used to analyze data that is assumed to be collected using a simple random sampling scheme. These methods, however, tend to underestimate variance when the data is collected with a cluster design, which is often found in educational survey research. The purposes of this paper are to demonstrate how a cluster design affects the standard error statistic and the subsequent analyses, and to present practical techniques to analyze data from cluster designs correctly. A heuristic example is given to illustrate how to compute the variance estimate for a cluster design and the corresponding design effect. Simulation data is then used to examine variance estimation results from oneand two-stage cluster designs, respectively. Both a formula approach and a jackknife resampling approach are used in obtaining variance estimates. It is shown that, for 150 observations sampled from a population of 1,000, using a 2-stage cluster design, the actual variance can be underestimated by a factor of 3 if the standard statistical method is used. The underestimated variance or standard error statistic will lead to unwarranted statistical significance in hypothesis testing, or a narrow confidence interval in parameter estimation. Consequently, misleading conclusions can be made based on these inappropriate analysis findings. (Contains 1 tables and 11 references.) (Author/SLD) ******************************************************************************** * Reproductions supplied by EDRS are the best that can be made * * from the original document. * ******************************************************************************** PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY WOA TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) is document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. THE EFFECT OF CLUSTER SAMPLING DESIGN IN SURVEY RESEARCH ON THE STANDARD ERROR STATISTIC

The jackknife and bootstrap methods are becoming increasingly popular in research. Although the two approaches have similar goals and use similar strategies, information is lacking with regard to the comparability of their results. In the present study, this issue was systematically investigated for a case of canonical correlation analysis. Bootstrap, jackknife, and Monte Carlo experiments were carried out for 4 sample sizes (n = 200, 100, 50, 20), The jackknife analyses were also varied as regards the number of jackknife observations deleted in each analysis, Some meaningful discrepancies were observed between the bootstrap and jackknife results, especially under small sample-size conditions, Based on the comparisons made with Monte Carlo estimates, the empirical results suggest that the bootstrap technique provides less biased and more consistent results than the jackknife technique does.

Research in covariance structure analysis suggests that nonnormal data will invalidate chi-square tests and produce erroneous standard errors. However, much remains unknown about the extent to and the conditions under which highly skewed and kurtotic data can affect the parameter estimates, standard errors, and fit indices. Using actual kurtotic and skewed data and varying sample sizes and estimation methods, we found that (a) normal theory maximum likelihood (ML) and generalized least squares estimators were fairly consistent and almost identical, (b) standard errors tended to underestimate the true variation of the estimators, but the problem was not very serious for large samples (n = 1,000) and conservative (99%) confidence intervals, and (c) the adjusted chi-square tests seemed to yield acceptable results with appropriate sample sizes.

The jackknife and bootstrap methods are becoming more popular in research. Although the two approaches have similar goals and employ similar strategies, information is lacking with regard to the comparability of their results. This study systematically investigate,1 the issue for a canonical correlation analysis, using data from four random samples from the National Education Longitudinal Study of 1988. Some conspicuous discrepancies are observed mainly under small sample size conditions, and this raises some concern when researchers need to choose between the two for their small samples. Due to the lack of theoretical sampling distributions in canonical analysis, it is unclear which method had superior performance. It is suggested that Monte Carlo simulation is needed for this kind of comparison. It is also suggested that caution is warranted in generalizing the results to other statistical techniques, since the validity of such generalizations is uncertain. (Contains 6 tables and 18 references.) (Author/SLD)*********************************************************************** Reproductions supplied by EDRS are the best that can be made * from the original document. *********************************************************************** * U S DEPAKTMENT OF EDUCATION Ofl)ce of Educid)onsf ROSCIrCh and improvernni EDUCATIONAL RE SOURCES INFORMATION CENTER (ERIC) TP.s docurnem! Pis beer) reproduced as ,ece.ved from Ihe person or ordsrutstoon or.grnat.op .1 o Manor changes hve been made to approve reproduct)on dowdy Po)nts of rrew opostons slated .ri pus docu. map? do not neCessanly rety.senl ottrcrli OEM 1)03.1.01, or pohcy Jackknife and Bootstrap 1 PERMISSION TO REPRODUCE THIS MA TERIAL HAS BEEN GRANTED BY X 1_1/96 F inJ TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) HOW COMPARABLE ARE THE JACKKNIFE AND BOOTSTRAP RESULTS: AN INVESTIGATION FOR A CASE OF CANONICAL CORRELATION ANALYSIS Xitao Fan Utah State University

AUTHOR Fan, Xitao; Wang, Lin TITLE Practical Guidelines for Evaluating Sampling Designs in Survey Studies. PUB DATE 1999-11-00 NOTE 25p.; Paper presented at the Annual Meeting of the American Evaluation Association (14th, Orlando, FL, November 3-6, 1999). PUB TYPE Guides Non-Classroom (055) Speeches/Meeting Papers (150) EDRS PRICE MF01/PC01 Plus Postage. DESCRIPTORS *Evaluation Methods; *Research Design; Research Methodology; *Sampling; *Surveys