Conducting behavioral analysis and administering questionnaires at the start of psychotherapy is time consuming and costly. This study explores a new approach linking diagnostic information across disorders to behavioral response categories (BRCs) used in behavioral analysis addressing the research gap concerning the integration of behavioral analysis into questionnaire-based diagnostic methods. Clinical psychologists assigned SCL-90-R items to one of four response variables (emotional, cognitive, physiological, behavioral). Three groups emerged: items unequivocally assigned to one BRC, items assigned to multiple BRCs (combined M var = 296.15), and items that couldn't be assigned due to expert disagreement, ambiguous formulations, or enumerations in items ( M var = 109.97). The feasible approach enables receiving multiple types of diagnostic information while only executing one questionnaire, potentially enabling cost-effective diagnostics, reducing therapy time, and making treatment more accessible to individuals with lower socioeconomic backgrounds. The findings emphasize the importance of semantic clarity in item formulations.
This study addresses enrollment and compliance in ecological momentary assessment (EMA) research by focusing on a sample of older adults with varying levels of technological familiarity. This is noteworthy because contemporary EMA studies predominantly require participants to use smartphones to record data in real time. We employ a two-stage analysis where we estimate the probability of enrollment based on sociodemographic and research design attributes and next use the output from this model to address potential selection bias in a subsequent model estimating compliance rates. We find that smartphone ownership is a major predictor of enrollment and that offering participants a loaner smartphone is insufficient to overcome this barrier. Ultimately, our findings suggest that while there are some potential design strategies that prospective researchers could implement to improve enrollment and compliance, there will likely always be some people whose attributes decrease their probability of enrollment and compliance regardless of intervention.
The growing work on causal modeling has produced as one key insight that we should not give a causal interpretation to estimates of controls. We should only interpret the treatment effect of interest. I show that this guidance that is valid for treatment effects is not necessarily valid for case selection in mixed-methods research when cases are selected based on their residuals. The effect of a control variable needed to identify the treatment effect may not itself be identified. While unproblematic for treatment effect interpretation, the resulting bias undermines the role that residuals are supposed to play for case selection. Guided by the distinction between good, bad, and neutral controls, I explain what identification assumptions must be met in mixed-methods research to use residuals for case selection. I present practical guidance on how to perform residual-based case selection in mixed-methods studies.
Research with families and friends of homicide victims (FFHV) requires researchers to consider how to best protect participants. Due to the nature of this type of research, researchers need to be aware of concerns such as sensitivity due to the emotional weight of sharing about their loved one, mental health, and trauma. Although there has been an increase in research focused on FFHV, no known research has provided guidance on considerations for holding focus groups (FG) with FFHV. Based on several years of holding focus groups with FFHV, this short take provides suggestions that include the following: selecting a research team; confidentiality; victim-informed questions; the research environment; research time and location; mental health concerns; participant burden; support information; concluding the groups; and researcher transparency and accountability.
I examined behavioral changes in web surveys, focusing on shifts in attentiveness over time. Using response-time Gaussian mixture modeling, respondents were classified into three types: consistently inattentive, deteriorating (initially attentive but later disengaged), and consistently attentive. For the deteriorating group, a Weibull model estimated the timing of lapses relative to cumulative time, revealing a sharp decline in attentiveness at seven-nine minutes. These findings challenge static views of inattentiveness as a fixed trait and support a dynamic framework of attentional decline during survey participation. By quantifying when and how inattention occurs, this approach provides insights into inattentive responding patterns and supports real-time monitoring and adaptive design.
The debate on the need to consider the relative importance of various life domains (such as health and finances) in linking domain-specific satisfaction to subjective well-being (SWB), often known as domain importance weighting, has been ongoing. One major issue facing researchers who study domain importance weighting in the context of SWB is that there is no consensus on the weighting function of domain importance. Without clear a priori knowledge of the weighting function of domain importance, assessing the appropriateness of domain importance weighting in the SWB literature has been limited. This article investigates domain importance weighting in the context of SWB without assuming any specific weighting function of domain importance.
360-degree photography is an emerging form of visual media that is increasingly used in research. However, little methodological evidence exists on the use of this technology for field-based data collection and analysis. Drawing on an "urban objects" research project, we describe procedures to capture, process, store, code, and analyze 360-degree digital photos, highlighting considerations particular to this imaging format. 360-degree photography is a powerful field method, particularly for research that requires "whole scene capture" and/or strives to analyze the relational connections between objects and their wider surrounding environments. The procedures outlined are applicable to field-based research projects across the social sciences.
This article examines how positionality and various researcher–respondent combinations enhance our ability to gain holistic representations of the ethnically diverse Qatari society. While there has been a substantial growth in the use of qualitative research methods in the Arab World since the 1990s, methodological analyses of this qualitative shift remain scarce. Our two years of extensive research on the National Museum of Qatar involving 135 semi-structured interviews and six focus groups conducted by a diverse research team with a diverse population enabled us to identify practical guidance on the employment of qualitative research methods in Qatar, and by extension to the socially, culturally, and politically similar wider Arab Gulf region.
This Short Take describes the rationale for, and use of, a decision-making process in community-engaged comparative effectiveness research, co-led by community researchers and academic researchers. The contribution of this methodological guide is to (1) ensure that all partners have a voice in contributing to major study decisions; (2) distribute power equally; (3) avoid power differentials; (4) increase trust; and (5) promote shared governance in research.
Prepaid monetary incentives increase response rates in surveys. Whether offering a second monetary incentive to nonrespondents in web-and-mail surveys increases response rates and improves sample composition is relatively unexplored. This article examines an experiment in a general population probability survey in which nonrespondents were randomly assigned to receive $0, $1, $2, or $5 in a later mailing after receiving an initial $1 in the first mailing. Response rates increased for the sequential incentive conditions, but at a decreasing rate. There was no consistent difference in sample composition across the sequential incentive levels, although younger and more mobile adults were more likely to participate with sequential incentives. Costs per completion were similar for the $1 and $2 conditions ($51 and $54, respectively), but about $83 per complete for the $5 condition. Survey practitioners may consider using a $1 or $2 sequential incentive to increase response rates at a reasonable cost.
The increased use of experiments in social research yields more discussions and knowledge about the design and utility of the methods. Based on previous research and two empirical studies that apply a factorial survey experiment (FSE) and a field experiment, both aiming at detecting employers' recruitment behavior toward wheelchair users, I discuss the behavioral validity of an FSE and research ethics in both experiments. The results of the two experiments are highly comparable and indicate a high behavioral validity of the FSE in studies of recruitment behavior. While field experiments tend to enjoy precedence in the social sciences, they also involve serious ethical considerations. An FSE, on the other hand, is criticized for having a low behavioral validity but complies better with research ethics than field experiments. This study contributes with a nuanced understanding of the behavioral validity of an FSE in studies concerning hiring behavior.
We compared trust game behavior in three environments: (1) The standard laboratory with physically present participants (Laboratory); (2) An online environment with an online meeting before playing the trust game (Online plus a meeting); and (3) An online environment without a meeting (Online without a meeting). In Laboratory, participants convened in a classroom and interacted via computers. In Online plus a meeting, participants could see each other in an online meeting before sessions started. In Online without a meeting, participants did not see each other during the experiment. The experimental design enables examination of the influence of the environment on choice behavior. Participants in all conditions were recruited through the same pool of subjects. We did not see a statistically significant difference between the treatment conditions regarding trust or trustworthiness. Our results suggest that online experiments will yield similar results with experiments implemented in a laboratory.
In-situ measurements, using the experience sampling method (ESM), can provide insight into behaviors and contextual factors by allowing individuals to self-report them via text or push messages on a smartphone close to the behavior of interest. However, more is needed to know about the data quality of these measures, particularly the impact of sampling frequency. This study aims to examine the effects of different sampling frequencies on compliance, sample biases, and reactivity of measures in the context of digital media use. In July 2021, a group of Dutch citizens (n = 250) was randomly assigned to either a standard daily-intensive burst measure (DI-BM; seven surveys across the day) or hourly-intensive burst measure (HI-BM; 12 surveys over two hours per day) condition and surveyed across seven consecutive days, resulting in a total number of 16,135 surveys sent. Results indicate higher compliance in the standard ESM condition than in the burst ESM condition.
Past personal network research has highlighted the many trade-offs in data collection strategies but has largely overlooked respondents’ perspectives on the interview experience. We use a within-subjects research design to compare respondents’ experiences with pen-and-paper sociograms and a comparable computer-assisted approach using Network Canvas, amongst young people in rural KwaZulu-Natal, South Africa. Data was collected through five focus groups (31 participants in total) and analyzed using thematic analysis and a novel journey-based approach. Our results highlight key experiential trade-offs between the analog and digital methods focusing on cognitive burden, emotional investment, and practical constraints. Centrally the analog method allowed greater emotional investment, which may or may not be desired. This work showcases the merits of a journey-based analytical approach for better understanding the interview process from respondents’ point of view, and highlights that considering sociogram tools experientially invites greater focus on ethical concerns for respondent care.
This article investigates cognitive load in fully labeled and end-labeled Likert-type scales. We compared cognitive load between two conditions using different methods such as pupillometry, completion times, and subjective evaluation of mental effort. We conducted a laboratory experiment (N = 170) using eye-tracking technology to measure pupil dilation while respondents completed a web survey. The mean pupil diameter was larger in the fully labeled than in the end-labeled condition. Because the fully labeled scales provide more information and require respondents to read labels for all scale values, they increase neurophysiological load compared to the end-labeled scales. Though the effect was small, it was consistent. We found no difference in completion time and subjective rating of mental effort between the conditions. Despite higher cognitive load in the fully labeled condition, data quality benefits from the provision of clear verbal labels in the scale.
During the COVID-19 pandemic, face-to-face surveys changed their mode of data collection due to contact restrictions and quarantine regulations. We report the experiences from the fifth wave of our panel in Cologne, Germany (n = 915). Data collection took place after the peak of the pandemic (summer 2022), maintaining the face-to-face mode of previous waves, with 30% of participants opting for the interviewer to wear a face mask. We find that a preference for face mask usage is not linked to the sociodemographic characteristics of participants. Interviewers reported feeling good and safe conducting face-to-face interviews despite potential infection risks. We also find no effects on item nonresponse and length of interview. And even though the open-ended responses decreased in length, their qualitative content was not affected by the face masks. These results provide a positive outlook for researchers planning to conduct face-to-face interviews in challenging times or with at-risk populations.
Translating questionnaires is challenging when the source language structure lacks equivalents in the target language. Using item 8 ("I wish I could have more respect for myself") of the Rosenberg Self-esteem Scale, we experimentally compared three strategies for translating the subjunctive mood from English to Chinese: (1) a close translation preserving vocabulary and structure; (2) an adaptive translation explicitly expressing the self-esteem deficiency; and (3) a problematic yet widely used translation. Results showed the adaptive translation yielded better psychometric properties than the close translation and increased respondent alertness through explicit negative wording. Our findings suggest that the issues with item 8 in non-English samples likely stem from inappropriate translation rather than cross-cultural differences.
The free-list method has enjoyed a remarkably productive history, yet most free-list research is limited to informal comparisons that are heavily reliant on point estimates such as item or cultural salience. Here, we demonstrate a range of methods to incorporate uncertainty into such group-level estimates. This approach involves: (1) resampling individual-level data (via bootstrapping or Bayesian regression) to create a range of hypothetical alternative samples; (2) generating group-level estimates (e.g., Smith's S) in each sample; and (3) using the variation in these estimates as uncertainty intervals. While we focus predominantly on cultural salience, this approach can be applied to other free-list metrics. We also present some extensions to this approach, such as comparing estimates between items and between groups. We provide open data and code to help readers gain familiarity with these methods. Ultimately, we encourage researchers using free-list data to move beyond simply reporting point estimates.
QR codes are increasingly used in postal invitations to web surveys to facilitate participation. We experimentally tested how a QR code in the invitation letter affects participation in a push-to-web survey of the German family panel (pairfam). The response rate was three percentage points higher with QR codes than without. The QR code effect on response did not differ by gender or education, but it was significantly higher for middle-aged than for younger individuals. Additionally, the proportion of respondents using mobile devices to complete the survey increased substantially. This increase was exclusively due to a 12-percentage-point higher proportion of smartphones, with no increase in the proportion of tablets.
In rural sub-Saharan African communities, researchers often struggle to balance active participation with respect for community members' livelihood time. This article reflects on my step-by-step fieldwork experience with Sharing Circles, a method that proved effective in facilitating meaningful community engagement without imposing on participants' daily routines. Unlike conventional approaches that may disrupt agricultural or family responsibilities, Sharing Circles provide a respectful and inclusive space that allow participants to contribute without compromising their productivity.
