Acquiescent response style (ARS) is the tendency to agree with measurement items regardless of their content. ARS tends to differ across cultures, making it particularly relevant for cross-cultural research, as it may disrupt cross-cultural comparability. Balanced measurement scales include items written in different directions of the measured construct and are hypothesized to be a remedy for undesirable effects of ARS. This study examined the effect on measurement properties of three scale balancing approaches in a cross-cultural research setting: (i) unbalanced scales; (ii) scale balancing with negated items where some of the items were written with a negation, such as "no" or "not"; and (iii) scale balancing that introduce items written with polar opposite terms (e.g. "unhappy" as the opposite of "satisfied") that reversed the item direction without using negations. This study examined these approaches on four well-being measurement scales, originally unbalanced, that used an Agree-Disagree Likert response scale. Participants were recruited to a Web survey from three groups: U.S. non-Hispanic whites (n = 1,200, interviewed in English), U.S. Hispanics (n = 1,200, interviewed in English), and Mexicans (n = 1,200, interviewed in Spanish). Respondents were randomly assigned to one scale-balancing conditions for well-being measures. Balanced scales outperformed unbalanced scales for convergent validity, with higher correlations observed between scale scores and validation measures. Between negated and polar opposite balancing, no statistical differences were observed in factorial model fit, reliability, or convergent validity. This study supports that, when designed carefully, balanced measurement scales may be effective for addressing ARS and improving measurement properties over unbalanced scales.
Respondent-driven sampling (RDS) is widely used to recruit hard-to-sample populations, including racial and ethnic minority groups, yet limited evidence exists on how seed selection mechanisms affect data quality. This study examines seed selection in the Health and Well-Being of Koreans (HAWK) study, a national web-based RDS survey of Korean American adults. We experimentally varied seed recruitment using two mechanisms: a probability-oriented postal address list associated with Korean ethnicity or surnames (“mail seeds”) and convenience-based online recruitment through platforms such as Facebook (“non-mail seeds”). The final sample included 772 Korean American adults from 57 seeds. We evaluated geographic coverage and benchmarked HAWK estimates against 2022 American Community Survey (ACS) estimates for 18 demographic, socioeconomic, immigration, health care access, and disability variables. We also assessed whether statistical adjustment methods improved the quality of HAWK estimates. RDS expanded the geographic coverage of the sample, from seeds located in 25 states to respondents in 36 states. However, data quality varied by seed selection mechanism. Samples generated from mail seeds more closely approximated ACS benchmarks than samples generated from non-mail seeds, particularly for age, education, nativity, citizenship, and immigration-related measures. Statistical adjustments generally increased standard errors without improving bias reduction. Findings suggest that Web-RDS can effectively reach geographically dispersed racial and ethnic minority populations, but that seed selection has nontrivial consequences for data quality. Probability-oriented seed recruitment may improve data quality more effectively than statistical adjustments. Future RDS studies should give greater attention to seed recruitment mechanisms.
This study extends the scope of the paradata discussion to respondent-driven sampling (RDS). Unlike traditional sampling, RDS relies on existing social networks within a target population. This unique process provides opportunities to produce novel paradata. Specifically, this study examined two types of paradata in RDS: one based on interviewer observations and the other based on recruitment behaviors ascertained from tracking recruitment coupons. We implemented these paradata features in two independent RDS surveys. In an in-person RDS survey of persons who inject drugs in Southeast Michigan, we implemented an interviewer observation questionnaire. This included questions about interviewers' assessments of respondents' understanding of coupon distribution instructions, as well as their expectations regarding respondents' chances to recruit others and to return for a follow-up interview. These observations predicted recruitment success. In a Web-RDS study of Korean Americans, physical distance between linked respondents (such as a respondent and their recruiter) was determined by tracking recruitment coupons and geocoding respondent addresses. Greater geographic distance was associated with a higher likelihood of serious psychological distress. The results demonstrate that the unique features of RDS offer new avenues for utilizing paradata in both methodological and substantive research. These findings warrant further exploration and development of paradata specific to RDS.
Social networks are the fundamental premise of respondent driven sampling (RDS). Personal network sizes termed as “degrees” play an important role in the RDS literature, as RDS-specific point estimators incorporate degrees as an adjustment factor approximating the selection probabilities. For this reason, degrees relevant to RDS should consider recruitability rather than the state of connectedness. This study examines various measures of degrees (standard degrees; degrees with priming recruitment requests; and degrees using naming stimuli that tap into specifics of social relationships) in two independent RDS surveys: one targeting people who inject drugs (PWID, n=410) and the other targeting Korean immigrants (n=637). The latter randomized interview language for bilingual English-Korean speakers. There was greater noise in the standard degree measure compared to other degree measures. With a subtle hint about the recruitment request, respondents reported knowing fewer people, compared to the standard degree question, implying a mismatch between the standard degree question and recruitability. Degree reports were sensitive to interview language: reported degrees were smaller in Korean than English interviews and better explained the recruitment. Degrees measured in the contexts of close social relationships were shown to improve inference, while this was not true for the standard degree. This warrants scrutiny of network measures that reflect the RDS recruitment mechanism.
Respondent-driven sampling (RDS) is widely used to collect data from hidden populations in social and biomedical science. Although RDS may provide comprehensive coverage of the target hidden population through social network recruitment, its non-random sampling process poses challenges for generalizing findings beyond the sample. Current analytical methods rely on the network size (degree) reported by respondents to adjust for unequal sampling probabilities. However, the accuracy of the reported degree is questionable due to reporting errors, evidenced through an unusual frequency of multiples of five and improbably large values. To address this measurement error, we leverage a byproduct of the RDS process (e.g., respondents' recruitment patterns) and develop a novel degree estimator based on a latent variable model of the true degree that accounts for response errors via a reporting mechanism and incorporates recruitment information and external demographic profiles. The effectiveness of the proposed method is demonstrated through a case study and a simulation study, which shows accurate and reliable degree estimates leading to significant improvements in population parameter estimation.
Introduction Polysubstance use (PSU), particularly opioid-involved and stimulant-involved PSU, is a growing issue in the USA. PSU increases the risk of negative health consequences, including infectious diseases, worsening physical and mental health conditions, and overdose-related deaths. These consequences occur in the context of varying health risk behaviours, substance-related preferences, and treatment engagements among people with PSU. To inform improvements in prevention, harm reduction, and substance use disorder (SUD) treatment, additional research is needed to comprehensively understand the current context and drivers of PSU preferences, motivations, and behaviours.Methods and analysis Herein, we describe the protocol for a prospective cohort study designed to capture detailed patterns, profiles, and trajectories of PSU, with the aim of comprehensively examining the drivers of PSU behaviours and SUD treatment utilisation. Adults (ages 18–75; n=400) who engage in PSU will be recruited from healthcare institutions, an established participant database maintained by an adjacent SUD research team, and online advertisements. Study assessments will capture dynamic patterns, choice preferences, and motivators of PSU via behavioural economic (BE) measures, detailed Timeline Follow-Back (TLFB) interviews, and self-administered surveys. The assessment timeline will include a baseline survey and TLFB interview, weekly TLFB interviews for 4 weeks post-baseline, and follow-up surveys and TLFB interviews at 4-, 8-, and 12-months post-baseline.Ethics and dissemination The study is funded through the National Institutes of Health Helping to End Addiction Long-term (HEAL) initiative and was approved by the University of Michigan Medical Institutional Review Board. Findings will be disseminated to academic, clinical, and community partners through the Michigan Innovations in Addiction Care through Research and Education programme. Results from this study will inform actionable and practical insights relevant to the delivery of personalised care in the context of PSU.
This Guide to Statistics and Methods provides an overview of weighted analyses of population-based survey, which can help achieve statistically valid, representative population-based findings.
Many surveys target population subgroups that may not be readily identified in sampling frames. In the case study that motivated this study, the target population was households with children between the ages of 3 and 10 from two areas surrounding Cleveland, Ohio and Dallas, Texas. A standard approach is to sample households from these two areas and then screen for the presence of age -eligible children. Based on the estimated number of age -eligible households in these two areas, this approach would have required completing screening interviews with 5.4 to 5.7 households to find one eligible household. We developed a model -assisted sample design strategy to improve screening efficiency by attaching a measure of eligibility propensity to each household in the population. For this, we used a modeling and imputation strategy that combined information from several data sources: (1) the population of addresses for these two areas with demographic covariates from a commercial vendor, (2) external population data (from the American Community Survey and Census Planning Data) for these two areas, and (3) screening data from a large nationally representative survey. We first tested this sampling strategy in a pilot study and then implemented it in the main study. This strategy required 4.2 to 4.3 completed screeners to identify one eligible household. The proposed approach therefore improved the sampling efficiency by about 25% relative to the standard approach.
Acquiescent response style (ARS), the tendency for survey respondents to agree with survey items, is of particular concern for increasing measurement error in surveys with populations who are more likely to acquiesce, such as Latino respondents in the U.S. In order to develop methods for reducing ARS, this study addressed two questions: (1) Does administering a questionnaire using conversational interviewing (CI) yield less ARS than standardized interviewing (SI)? (2) Do bipolar disagree/agree (DA) response scales lead to higher ARS than unipolar response scales that do not assess agreement (non-AG)? A total of 891 Latino telephone survey respondents were screened for ARS and randomly assigned to four experimental groups determined by crossing interviewing technique (CI or SI) and response format (non-AG or DA): (1) SI/non-AG ( n = 301); (2) SI/DA ( n = 295); (3) CI/non-AG ( n = 149); and (4) CI/DA ( n = 146). CI yielded lower ARS than SI ( p < 0.001), but there was no difference in ARS between DA and non-AG response scales. A subset of coded interview recordings indicated that the CI interviewers reduced ARS by clarifying questions even in the absence of evidence of respondent confusion and helping with response mapping. These results suggest that difficulty answering questions associated with cognitive decline and cultural norms may have prompted higher use of ARS, but that conversational interviewers were able to mitigate these difficulties and cultural tendencies. Findings from this study suggest that using CI to administer survey questions may decrease ARS and improve data quality among survey respondents who are more likely to engage in ARS.
Acquiescent (ARS) and extreme response styles (ERS) can have detrimental effects on survey data and, for unknown reasons, are more frequently used by Latino than non-Latino white respondents. This exploratory study examined the influence of culture on these response styles by investigating their associations with individual-level cultural factors and ARS and ERS among 1,296 Mexican American, Puerto Rican, and Cuban American telephone survey respondents. Principal components representing stronger endorsement of marianismo/ machismo and social attentiveness ( simpatía, personalismo, respect for elders, value for sincerity, collectivism, individualism) were associated with higher ARS and ERS, while higher trust in strangers and more limited health literacy were associated with lower ERS. Findings from this study will enable survey designers to better anticipate ARS and ERS in surveys with Latino populations and, in turn, guide the selection of data collection and analysis methods to mitigate measurement error in the presence of these response styles.
The anchoring vignette method is designed to improve comparisons across population groups and adjust for differential item functioning (DIF). Vignette questions are brief descriptions of hypothetical persons for respondents to rate. Although this method has been adopted widely in health surveys, there remain challenges. In particular, vignettes are complex, increasing survey time and respondent burden. Further, the assumptions underlying this method are often violated. To overcome such challenges, this paper introduces an innovative technique, namely image anchoring vignettes, conveying vignette information with varying health levels in images. We conducted a cross-cultural experimental study to examine the performance of image and standard text vignettes in terms of response time, how well they satisfy the assumptions, and their DIF-adjusting quality using a confirmatory factor analysis. The study revealed that respondents can better differentiate the intensity levels of the three vignettes in the image vignette condition, compared to text vignettes. Response consistency assumption appears to be better satisfied for image vignettes than text vignettes. Using well-designed image vignettes greatly reduces survey time without losing the DIF-adjustment quality, indicating the potential of image vignettes to improve overall efficiencies of the anchoring vignette method. Improving vignette equivalence (i.e., minimizing different interpretations of vignettes by different groups), remains a challenge for both text and image vignettes. This study generates new insights into the design and use of image anchoring vignettes.
Objectivesativity and family support may influence attitudes and behaviors that delay or accelerate the disability process in older adults. The objectives of this study were twofold: 1) to evaluate nativity and migration cohort differences in trajectories of disability (assessed by activities of daily living [ADL]) among older Mexican Americans; and 2) to determine the role of objectively measured family support in the association between nativity, migration cohort, and disability changes over time.MethodsThis is a longitudinal study with up to 18 years follow-up (1993-2011) using data from the Hispanic Established Populations for the Epidemiologic Study of the Elderly (N=2,785, mean age =72.4 years). Disability was assessed using self-reported limitations in activities of daily living (ADL). Nativity and migration cohort were self-reported. Family support was assessed by marital status and the number of their children participants saw each month. Linear growth curve models evaluated the trajectory of ADL disability over 18 years and assessed variations by nativity status, migration cohort and family support.ResultsForeign-born respondents who migrated before age 20 had more starting ADL limitations (β= .36, P<.001) and accumulated disability faster (β=.04, P<.01) compared with their US-born counterparts. In contrast, foreign-born respondents who migrated at later ages showed disability trajectories similar to US-born respondents. Married respondents had a lower level of disability (β= -.14, P<.01) and a lower rate of accumulation over time (β= -.02, P=.001) compared with participants who were not married.DiscussionMexican Americans who migrate at younger ages may experience greater disability over time; however, family support may help mitigate the accumulation of disability among older Mexican Americans.
This study examined feasibility and methodological utilities of respondent driven sampling (RDS) for Korean immigrants. We conducted the Health and Life Study of Koreans (HLSK), a Web-based RDS study targeting foreign-born Korean Americans. Through chain referrals, n = 638 participated. Geographic coverage and estimates of HLSK were compared to foreignborn Korean samples in the American Community Survey and the California Health Interview Survey as benchmarks. Compared to the benchmarks, HLSK fared well on the geographic coverage, household type and size, employment and health insurance but over-captured those who were younger, more recent immigrants, with higher education and with disability. Existing RDS-specific estimators were largely ineffective. Conclusions. RDS may serve as a cost-effective tool for recruiting recent immigrants, a harder-to-recruit subgroup within minorities. However, recruitment noncooperation posed operational challenges, a critical gap in the literature. This leaves RDS yet to be a reliable methodology.
Question What longitudinal changes in self-reported health status and days of poor health among racial, ethnic, urban/rural, and very-low-income subgroups of enrollees are associated with Medicaid expansion? Findings In this survey study of 3097 respondents, reports of fair or poor health and days of poor physical health decreased over time among enrollees, especially among non-Hispanic black enrollees and those with very low incomes. There were no statistically significant differences in the number of days of poor mental health or the number of days of usual activities missed owing to poor physical or mental health over time. Meaning These findings suggest that within Medicaid expansion, the health of vulnerable populations is improving. This survey study examines the self-reported health of enrollees in Michigan's Medicaid expansion, the Healthy Michigan Plan, among respondents to surveys in 2016 and 2017. Importance Evidence about the health benefits of Medicaid expansion has been mixed and has largely come from comparing expansion and nonexpansion states. Objective To examine the self-reported health of enrollees in Michigan's Medicaid expansion, the Healthy Michigan Plan (HMP), over time. Design, Setting, and Participants A telephone survey from January 1 to October 31, 2016 (response rate, 53.7%), and a follow-up survey from March 1, 2017, to January 31, 2018 (response rate, 83.4%), were conducted in Michigan, which expanded Medicaid in 2014 through a Section 1115 waiver permitting state-specific modifications. Four thousand ninety HMP beneficiaries aged 19 to 64 years with at least 12 months of HMP coverage and at least 9 months in a Medicaid health plan were eligible to participate. Data were analyzed from April 1 to November 30, 2018. Main Outcomes and Measures Surveys measured demographic characteristics and health status. Analyses included weights for sampling probability and nonresponse. Comparisons between 2016 and 2017 included those who responded to both surveys (n = 3097). Results Of the 3097 respondents to the 2017 follow-up survey, 2388 (77.1%) were still enrolled in HMP (current enrollees) and 709 (22.9%) were no longer enrolled when surveyed (former enrollees). Among all follow-up respondents, a weighted 37.5% (95% CI, 35.3%-39.9%) were aged 19 to 34 years, 34.0% (95% CI, 31.8%-36.2%) were aged 35 to 50 years, and 28.5% (95% CI, 26.7%-30.3%) were aged 51 to 64 years; 53.0% (95% CI, 50.8%-55.3%) were female. Respondents who reported fair or poor health decreased from 30.7% (95% CI, 28.7%-32.8%) in 2016 to 27.0% (95% CI, 25.1%-29.0%) in 2017 (adjusted odds ratio [AOR], 0.66 [95% CI, 0.53-0.81]; P < .001), with the largest decreases observed in respondents who were non-Hispanic black (from 31.5% [95% CI, 27.1%-35.9%] in 2016 to 26.0% [95% CI, 21.9%-30.1%] in 2017; P = .009), from the Detroit metropolitan area (from 30.7% [95% CI, 27.0%-34.4%] in 2016 to 24.9% [95% CI, 21.6%-28.3%] in 2017; P = .001), and with an income of 0% to 35% of the federal poverty level (from 37.6% [95% CI, 34.2%-40.9%] in 2016 to 32.3% [95% CI, 29.1%-35.5%] in 2017; P < .001). From 2016 to 2017, the mean number of days of poor physical health in the past month decreased significantly from 6.9 (95% CI, 6.5-7.4) to 5.7 (95% CI, 5.3-6.0) (coefficient, -6.10; P < .001), including among current (from 7.0 [95% CI, 6.5-7.5] to 5.6 [95% CI, 5.1-6.0]; P < .001) and former (from 6.8 [95% CI, 5.9-7.7] to 5.8 [95% CI, 5.0-6.7]; P = .02) enrollees, those with 2 or more chronic conditions (from 9.9 [95% CI, 9.3-10.6] to 8.5 [95% CI, 7.8-9.1]; P < .001), across all age groups (19-34 years, from 4.3 [95% CI, 3.7-4.9] to 3.0 [95% CI, 2.5-3.5]; P < .001; 35-50 years, from 8.2 [95% CI, 7.3-9.0] to 6.9 [95% CI, 6.1-7.7]; P = .002; 51-64 years, from 9.0 [95% CI, 8.2-9.8] to 7.6 [95% CI, 6.9-8.3]; P = .001), and among non-Hispanic white (from 7.5 [95% CI, 7.0-8.1] to 6.1 [95% CI, 5.6-6.6]; P < .001) and black (from 5.9 [95% CI, 5.1-6.8] to 4.4 [95% CI, 3.6-5.1]; P < .001) respondents. No changes in days of poor mental health or usual activities missed owing to poor physical or mental health were observed. Conclusions and Relevance These findings suggest that HMP enrollees in Michigan have experienced improvements in self-reported health over time, including minority groups with a history of health disparities and enrollees with chronic health conditions.
Introduction: Michigan is one of 3 states that have implemented health risk assessments for enrollees as a feature of its Medicaid expansion, the Healthy Michigan Plan. This study describes primary care providers' early experiences with completing health risk assessments with enrollees and examines provider- and practice-level factors that affect health risk assessment completion. Methods: All primary care providers caring for >= 12 Healthy Michigan Plan enrollees (n=4,322) were surveyed from June to November 2015, with 2,104 respondents (55.5%). Analyses in 2016-2017 described provider knowledge, attitudes, and experiences with the health risk assessment early in Healthy Michigan Plan implementation; multivariable analyses examined relationships of providerand practice-level characteristics with health risk assessment completion, as recorded in state data. Results: Of the primary care provider respondents, 73% found health risk assessments very or somewhat useful for identifying and discussing health risks, although less than half (47.2%) found them very or somewhat useful for getting patients to change health behaviors. Most primary care provider respondents (65.3%) were unaware of financial incentives for their practices to complete health risk assessments. Nearly all primary care providers had completed at least 1 health risk assessment. The mean health risk assessment completion rate (completed health risk assessments/number of Healthy Michigan Plan enrollees assigned to that primary care provider) was 19.6%; those who lacked familiarity with the health risk assessment had lower completion rates. Conclusions: Early in program implementation, health risk assessment completion rates by primary care providers were low and awareness of financial incentives limited. Most primary care provider respondents perceived health risk assessments to be very or somewhat useful in identifying health risks, and about half of primary care providers viewed health risk assessments as very or somewhat useful in helping patients to change health behaviors. (C) 2019 American Journal of Preventive Medicine. Published by Elsevier Inc. All rights reserved.
This survey study assesses the association of Medicaid expansion in Michigan with enrollees’ employment or student status.