Objectives:The National Center for Health Statistics's Rapid Surveys System (RSS) was established in 2022. It uses two probability-based commercial online panels, NORC at the University of Chicago's AmeriSpeak panel and Ipsos's KnowledgePanel, to enable fast data collection on emerging or priority health topics. RSS Rounds 1-4 (RSS 1-4) were collected between August 2023 and May 2024. This report describes the methodology implemented in the early rounds of RSS for weighting and variance estimation, methods for combining multiple panels, and quality assessment. Methods:RSS 1-4 were weighted to a set of control totals for selected demographic and health variables from the National Health Interview Survey (NHIS). Estimates from the two online panels were combined using a convex combination based on effective sample sizes to produce one set of health estimates for release each round. As part of an ongoing quality evaluation, quality metrics were assessed and reported for each round and were used to inform the RSS methodology in successive rounds. Results:RSS 1-4 had sample sizes ranging from 7,046 (Round 2) to 8,705 (Round 4), with cumulative response rates ranging from 3.8% to 9.9% across the two panel providers. Quality metrics including nonresponse rates and breakoff rates were evaluated and found to be consistent for the two panel providers. In addition, a set of benchmark variables collected on RSS and NHIS were evaluated to assess bias. While the magnitude of bias varied by health variable and across health domains, most health variables had low bias across the four rounds. Conclusion:The methodology for RSS has evolved and allowed for changes to improve the quality of the program and the resulting estimates. This report describes the early methodology implemented in the first four rounds of RSS.
Objective:This report compares results between survey analysis software and the National Cancer Institute Joinpoint Regression Software for trend analysis of survey data. The "National Center for Health Statistics Guidelines for Analysis of Trends" recommends that analysts use record-level data and survey analysis software to fit desired trend models of survey data. When changes in a trend will be assessed using piecewise regression models, the guidelines recommend that analysts use the most recent version of the Joinpoint software with aggregated data to identify the number and location of joinpoints, then survey analysis software with record-level data to obtain final slope estimates and to conduct tests of hypothesis for the model identified by the Joinpoint software. In practice, the Joinpoint software sometimes produces results that appear incongruent with those generated by the survey analysis software. The purpose of this report is to provide guidance for conducting trend analyses on NCHS survey data to handle and explain these apparent inconsistencies. This report should be considered a supplement to the "National Center for Health Statistics Guidelines for Analysis of Trends." Methods:Cases were identified where apparent inconsistencies between the Joinpoint software and survey analysis software could occur. Plausible explanations for the apparent differences are discussed through text, examples, and frequently asked questions. Solutions are not provided; rather, recommendations, cautions, and additional information are provided to assist analysts in making the best decisions for their analysis and data. Results:Most frequently, inconsistencies occur when the prespecified piecewise regression model provided by the Joinpoint software is estimated using survey analysis software and successive slopes are not statistically significantly different from one another, resulting in one or more joinpoints being removed from the final model. Potential explanations are divided into five main categories.
Background:Synthetic data has been gaining popularity in many fields as an approach to retain data utility (the validity of inference using synthetic data) and protect confidentiality. However, creating synthetic data for complex surveys remains a challenge. Methods:This research compared three approaches to incorporate survey design information (stratification, clustering, and sampling weights) during the synthetic data-generating process using the Research and Development Survey (RANDS), a series of primarily web surveys conducted by the National Center for Health Statistics, Centers for Disease Control and Prevention. Both parametric (logistic and linear regression models) and nonparametric (classification and regression trees [CART]) methods were used to create synthetic data. Data utility and disclosure risk were evaluated via confidence interval overlap, propensity score measurement, and average matching probability for re-identification. Results:Using the original survey design information as predictors during the synthesis process improved data utility for the parametric method. However, the nonparametric method yielded results with better data utility but slightly higher disclosure risk.
Introduction:The Healthy People initiative provides science-based, 10-year public health objectives and targets for the U.S. population. As in the previous four initiatives, Healthy People 2030 established overarching goals and objectives (with targets) at the start of the decade and will be monitoring progress toward the attainment of targets and elimination of health disparities among population subgroups over the course of the decade. Objective:This report outlines Healthy People 2030 measurement practices for both progress toward target attainment and elimination of disparities and compares the 2030 measurement practices with those that were in place in 2020, highlighting strengths and limitations. Methods:Progress toward target attainment is assessed for the total population. The "percentage of targeted change achieved" quantifies movement toward targets, and the "percentage change from baseline" can be calculated for all core objectives. Based on the percentage of targeted change achieved or percentage change from baseline, as well as the statistical significance of these measures (when applicable), core objectives in Healthy People 2030 are classified into four mutually exclusive categories: TARGET MET OR EXCEEDED, IMPROVING, LITTLE OR NO DETECTABLE CHANGE, or GETTING WORSE. Disparities at a single timepoint are assessed by a suite of six measures: the between-group rate difference and ratio; summary rate difference and ratio; and maximal rate difference and ratio. To enable comparisons among those six measures, changes in disparities over time are assessed using the percentage change from baseline. Variability (standard errors and 95% confidence intervals) and statistical significance for all six measures, when applicable, are derived using a resampling/bootstrap procedure. Conclusion:Expanding and building on the approaches to measurement in previous decades, methods to measure progress toward target attainment and elimination of health disparities in Healthy People 2030 represent a further evolution of these methods and address methodological issues and limitations previously identified.
Objectives The Research and Development Survey (RANDS) is a series of web-based, commercial panel surveys that have been conducted by the National Center for Health Statistics (NCHS) since 2015. RANDS was designed for methodological research purposes,including supplementing NCHS' evaluation of surveys and questionnaires to detect measurement error, and exploring methods to integrate data from commercial survey panels with high-quality data collections to improve survey estimation. The latter goal of improving survey estimation is in response to limitations of web surveys, including coverage and nonresponse bias. To address the potential bias in estimates from RANDS,NCHS has investigated various calibration weighting methods to adjust the RANDS panel weights using one of NCHS' national household surveys, the National Health Interview Survey. This report describes calibration weighting methods and the approaches used to calibrate weights in web-based panel surveys at NCHS.
