Open data are often regarded as an important step towards improving the reproducibility and transparency of educational science. Yet, data sharing remains rare, and without open data, statistical analyses often remain irreproducible. In this article, we provide an introduction to synthetic data, a statistical technique based on multiple imputation (MI) that can be used to create simulated copies of the data that can be shared even when the original data cannot. To this end, we discuss reproducibility-related challenges of synthetic data and outline different approaches for generating synthetic data, including conventional and data-augmented MI (DA-MI) approaches to synthetic data. Furthermore, we conducted a case study using data from the PISA 2018 study, in which we aimed to address several challenges with synthetic data in educational research, such as missing data, multilevel data, and complex sampling designs. Our results indicate that these challenges can be addressed with relatively simple tools and that synthetic data can reproduce the results in a variety of statistical analyses. Finally, we discuss remaining challenges and directions for future research.

Teacher burnout is assumed to impair the cognitive, motivational, and social functioning of teachers, thereby hindering their professional behavior. Although this is a rapidly growing field of research, a systematic research synthesis is still lacking. Therefore, we meta-analytically summarized primary studies on the work-related correlates of teacher burnout symptoms in terms of absenteeism, the quality of teacher-student interactions (i.e., emotional support, classroom management, and instructional support), and student motivation and achievement. Meta-analyses of 86 studies demonstrated negative associations between teacher burnout symptoms and teacher-student interactions, and student motivation and achievement, whereas absenteeism was positively related to burnout. Meta-regressions showed that the negative associations between teacher burnout, teacher-student interactions, and student motivation varied significantly depending on the rater perspective (i.e., teacher self-report vs external reports). Building on the meta-analytic findings, we propose directions for future research and highlight practical implications for intervention programs.

Missing data are a common challenge in multilevel designs, and multiple imputation (MI) is often used for handling them. Past research has shown that multilevel MI provides an effective treatment of missing data, so long as the imputation model takes the multilevel structure and the intended analyses into account, and modern methods have been developed that can accommodate even complex types of analyses. However, multilevel MI can be difficult to apply in practice, where the multilevel structure is often not very pronounced or not of immediate interest in the analysis. In these applications, existing methods can become unstable and often struggle to provide reliable results. In this article, we introduce a fully conditional specification (FCS) approach to multilevel MI that combines single-level imputation methods with group means (GM) or adjusted group means (AGM) to accommodate the multilevel structure. Based on theoretical investigations and multiple simulation studies, we evaluated the performance of these methods across balanced and unbalanced designs and with larger numbers of variables. Our findings suggest that the AGM approach - though not the GM approach - performs well across most scenarios we investigated and can even outperform conventional multilevel MI approaches in challenging applications. We also provide an illustrative example of implementing these methods in a simulated setting and discuss the implications of our findings for practice.

Missing data are common in longitudinal designs and are often addressed with multiple imputation (MI), either as single-level MI, which treats repeated measures as separate variables, or multilevel MI, which treats repeated measures as nested within participants. Previous research has shown that both approaches can perform well in time-structured designs but has largely focused on growth modeling applications, where the assumptions underlying multilevel MI were met. In the present article, we argue that single-level MI is a more flexible method for handling missing data in time-structured designs that requires fewer assumptions and can accommodate a wider range of analyses than multilevel MI. In this context, we also consider applications of single-level MI to longitudinal multiple-indicator designs, in which single-level MI can be extended with composite scores or dimension reduction techniques such as partial least squares (PLS) to accommodate the potentially large number of variables in these types of designs. Our results from two simulation studies suggest that single-level MI provides a flexible treatment of missing data and that PLS in particular can facilitate single-level MI in applications with many variables. We conclude by discussing implications for applied research and by illustrating the application of single-level MI in an empirical example.

Missing data frequently occur in longitudinal designs and are commonly addressed using multiple imputation (MI), either in the form of multilevel MI, which treats repeated measures as nested within participants, or single-level MI, which treats repeated measures as separate variables. Previous research has shown that both approaches can perform well in applications of latent curve models (LCMs) in which the assumptions underlying multilevel MI are met. Using two simulation studies, we extend prior work in two directions: (1) to applications of single-indicator LCMs in which the assumptions of multilevel MI are violated, and (2) to multiple-indicator designs, in which multilevel MI can be challenging to implement and single-level MI may be computationally unstable due to the large number of variables. Results indicate that single-level MI provides the most accurate results in the contexts studied here, especially in conditions in which common implementations of multilevel MI are misspecified, and that large numbers of variables can be accommodated in single-level MI by employing dimension reduction techniques such as partial least squares. We also discuss the implications of these findings for applied research and illustrate the application of these methods with an empirical example.

Background: While the efficacy of digital interventions for the treatment of depression is well established, comprehensive knowledge on how therapeutic changes come about is still limited. This systematic review aimed to provide an overview of research on change mechanisms in digital interventions for depression and meta-analytically evaluate indirect effects of potential mediators. Methods: The databases CENTRAL, Embase, MEDLINE, and PsycINFO were systematically searched for randomized controlled trials investigating mediators of digital interventions for adults with depression. Two reviewers independently screened studies for inclusion, assessed study quality and categorized potential mediators. Indirect effects were synthesized with a two-stage structural equation modeling approach (TSSEM). Results: Overall, 25 trials (8110 participants) investigating 84 potential mediators were identified, of which attentional (8 %), self-related (6 %), biophysiological (6 %), affective (5 %), socio-cultural (2 %) and motivational (1 %) variables were the scope of this study. TSSEM revealed significant mediation effects for combined self-related variables (ab = -0.098; 95 %-CI: [-0.150, -0.051]), combined biophysiological variables (ab = -0.073; 95 %-CI: [-0.119, -0.025]) and mindfulness (ab = -0.042; 95 %-CI: [-0.080, -0.015]). Meta-analytical evaluations of the other three domains were not feasible. Limitations: Methodological shortcomings of the included studies, the considerable heterogeneity and the small number of investigated variables within domains limit the generalizability of the results. Conclusion: The findings further the understanding of potential change mechanisms in digital interventions for depression and highlight recommendations for future process research, such as the consideration of temporal precedence and experimental manipulation of potential mediators, as well as the application of network approaches.

The efficacy of digital interventions for depression has been established. In contrast, only limited knowledge on their change processes is currently available, and precise effect size estimates for mediators are pending. This study aimed to systematically review mediation studies and meta-analytically evaluate indirect effects of cognitive and behavioral mediators in digital interventions for adults with depression. The databases CENTRAL, Embase, MEDLINE, and PsycINFO were systematically searched for eligible randomized controlled trials. Two independent reviewers extracted the data, assigned mediators to eight categories and evaluated the methodological quality of included studies. Two-stage structural equation modeling was applied to synthesize indirect effects for cognitive and behavioral mediators. Overall, 25 studies (8110 participants) were eligible, which investigated 31 cognitive, 29 behavioral and 24 other mediators. Meta-analyses yielded significant indirect effects for combined cognitive mediators (ab = -0.068; 95 %-CI: [-0.093, -0.047]; k = 14 studies) and combined behavioral mediators (ab = -0.037; 95 %-CI: [-0.048, -0.028]; k = 13), but not for the specific cognitive mediators interpretation bias and dysfunctional attitudes. The systematic review revealed that all studies fulfilled at least five out of nine methodological quality criteria for psychotherapy process research, but the risk of bias assessment raised some concerns, particularly in regard to potential deviations from intended interventions. Overall, the findings of this meta-analytic review contribute to the understanding of the mechanisms of change in digital interventions for depression, and can inform the evidence-based advancement of future interventions.

In recent years, psychological research has faced a credibility crisis, and open data are often regarded as an important step toward a more reproducible psychological science. However, privacy concerns are among the main reasons that prevent data sharing. Synthetic data procedures, which are based on the multiple imputation (MI) approach to missing data, can be used to replace sensitive data with simulated values, which can be analyzed in place of the original data. One crucial requirement of this approach is that the synthesis model is correctly specified. In this article, we investigated the statistical properties of synthetic data with a particular emphasis on the reproducibility of statistical results. To this end, we compared conventional approaches to synthetic data based on MI with a data-augmented approach (DA-MI) that attempts to combine the advantages of masking methods and synthetic data, thus making the procedure more robust to misspecification. In multiple simulation studies, we found that the good properties of the MI approach strongly depend on the correct specification of the synthesis model, whereas the DA-MI approach can provide useful results even under various types of misspecification. This suggests that the DA-MI approach to synthetic data can provide an important tool that can be used to facilitate data sharing and improve reproducibility in psychological research. In a worked example, we also demonstrate the implementation of these approaches in widely available software, and we provide recommendations for practice.

Research on the multimedia effect in testing indicates that static representational pictures (RPs) and, potentially, dynamic RPs that further subdivide the picture into segments may support students' mental processing. This might be especially relevant for mathematical word problems that pose high mental demands in a multistage solution process. Existing studies further indicate that contrary to expectations of practitioners, static decorative pictures (DPs) do not improve students' affective state. It is unclear if dynamic DPs that decorate each segment of a word problem better meet these expectations. In our preregistered online experiment that involved 308 students in a 3 x 2 mixed design, we manipulated word problems regarding three visualization conditions (RPs vs. DPs vs. text-only), and two kinds of dynamics (static vs. dynamic). As expected, static and dynamic RPs increased response correctness, metacognitive ratings, and satisfaction compared to text-only. Besides this replication and extension of the multimedia effect in testing, dynamic RPs did not outperform static RPs in direct comparisons, however. Both RP conditions did not extend the time-on-task compared to text-only. As expected, static DPs did neither increase response correctness nor metacognition nor satisfaction compared to text-only. However, dynamic DPs were also unable to increase satisfaction compared to text only. Additional analyses that took the item position into account unveiled that the time-on-task in dynamic DP items aligned to that of text-only items over the course of the experiment, so that the students might have ignored dynamic DPs over time. Finally, we conclude implications for using these visualizations in digital assessments.

Missing data are a common problem in longitudinal studies, and longitudinal designs pose unique challenges for the appropriate handling of missing data in occupational and organizational research. However, state-of-the-art methods such as multiple imputation (MI) are still used only sparingly in longitudinal studies, and the translation of methodological advice into practice can be challenging in these types of designs. The present article aims to guide researchers in the application of MI in longitudinal studies by providing an overview of the techniques and challenges associated with longitudinal MI as well as three step-by-step examples that focus on applications with different levels of complexity (e.g., scale- vs. item-level data) and different strategies for longitudinal MI (e.g., with single- vs. multilevel models in wide- and long-formatted data). In this context, we place a specific emphasis on flexible approaches to MI, which can be used with many different longitudinal designs and analyses, and we provide practical recommendations for common challenges. For each example, we provide extensive materials including the computer code, example data, and annotated versions of each example.

Multiple imputation (MI) is one of the most popular methods for handling missing data in psychological research. However, many imputation approaches are poorly equipped to handle large numbers of variables, which are a common sight in studies that employ questionnaires to assess psychological constructs. In such a case, conventional imputation approaches often become unstable and require that the imputation model be simplified, for example, by removing variables or combining them into composite scores. In this article, we propose an alternative method that extends the fully conditional specification (FCS) approach to MI with dimension reduction techniques such as partial least squares (PLS). To evaluate this approach, we conducted a series of simulation studies, in which we compared it with other approaches that were based on variable selection, composite scores, or dimension reduction through principal components analysis (PCA). Our findings indicate that this novel approach can provide accurate results even in challenging scenarios, where other approaches fail to do so. Finally, we also illustrate the use of this method in real data and discuss the implications of our findings for practice.

Multilevel and other types of dependent data are often incomplete, and the treatment of missing data can be particularly challenging in these types of data. The past years have seen a significant increase in both the number and scope of statistical methods for incomplete multilevel data, which includes imputation-based methods such as multiple imputation (MI) and model- based methods such as maximum-likelihood or Bayesian estimation (MLE or BE). The purpose of this chapter is to provide an overview of the different methods that have been recommended for handling missing data in multilevel analysis and to discuss the features of multilevel data that need to be considered when these methods are used. In this context, we discuss what options the different methods provide for accommodating the structure and analysis of multilevel data, and we illustrate their application in a series of simulated examples. Finally, we also review the availability of imputation- and model-based methods in statistical software and provide guidance for their application in practice.

The affective lives of adolescents are unique in that momentary affect in this age group is more negative and variable. This study examined how neuroticism and romantic relationships (i.e., relationship involvement and relationship quality) relate to adolescents' daily affective experiences. In a weeklong experience sampling period, 408 German adolescents (MAge = 16.83) reported up to five times per day on their positive and negative affect. We estimated mixed-effects location scale models to analyze interindividual differences in adolescents' affect level and variability. Adolescents with higher neuroticism experienced lower levels of positive affect, higher levels of negative affect, and higher variability of positive and negative affect. Whereas adolescents with a romantic partner did not differ from their single peers with regard to affect level, they experienced higher affect variability, although evidence for these effects was weak. Finally, among adolescents who were currently involved in a romantic relationship, those with higher relationship quality experienced more variability in their positive affect if they scored higher in neuroticism. Across models, effect sizes systematically differed between affect level and variability, positive and negative affect, as well as neuroticism facets. We discuss these findings in light of adolescents' affective dynamics, affective development, and personality-social relationship interactions.

Multilevel structural equation modeling (MSEM) is a statistical framework of major relevance for research concerned with people’s intrapersonal dynamics. An application domain that is rapidly gaining relevance is the study of individual differences in the within-person association (WPA) of variables that fluctuate over time. For instance, an individual’s social reactivity – their emotional response to social situations – can be represented as the association between repeated measurements of the individual’s social interaction quantity and momentary well-being. MSEM allows researchers to investigate the associations between WPAs and person-level outcome variables (e.g., life satisfaction) by specifying the WPAs as random slopes in the structural equation on level 1 and using the latent representations of the slopes to predict outcomes on level 2. Here, we are concerned with the case in which a researcher is interested in nonlinear effects of WPAs on person-level outcomes – a U-shaped effect of a WPA, a moderation effect of two WPAs, or an effect of congruence between two WPAs – such that the corresponding MSEM includes latent interactions between random slopes. We evaluate the nonlinear MSEM approach for the three classes of nonlinear effects (U-shaped, moderation, congruence) and compare it with three simpler approaches: a simple two-step approach, a single-indicator approach, and a plausible values approach. We use a simulation study to compare the approaches on accuracy of parameter estimates and inference. We derive recommendations for practice and provide code templates and an illustrative example to help researchers implement the approaches.

Students' interest is considered an important learning outcome, but it is also a relevant predictor for student learning, and future vocational choices. According to numerous studies, however, students' interest in STEM fields usually declines during the course of secondary education. From the perspective of science education, it is therefore necessary to foster or at least maintain students' interest. Despite the variety of approaches that have already been examined in order to promote student interest, the problem of low-interested students remains. Prior findings indicate that specific person characteristics and the students' perception of the situation seem to be moderate the effectiveness of many approaches. The current intensive repeated measure intervention study addresses the investigation of a possible interest trigger (formative assessment) and also the process that influences the perception of this trigger. Based on a sample of 9th-grade chemistry students (N = 200), three different interventions of formative assessment were implemented in regular classrooms. Students' situational interest was assessed repeatedly in short time intervals. Based on multilevel analyses, not all interventions were perceived as equally interesting by the students. While students' individual interest influences the perception of all interventions positively, the impact of gender, chemistry grade, and enjoyment varies across the interventions.

Likelihood ratio tests (LRTs) are a popular tool for comparing statistical models. However, missing data are also common in empirical research, and multiple imputation (MI) is often used to deal with them. In multiply imputed data, there are multiple options for conducting LRTs, and new methods are still being proposed. In this article, we compare all available methods in multiple simulations covering applications in linear regression, generalized linear models, and structural equation modeling (SEM). In addition, we implemented these methods in an R package, and we illustrate its application in an example analysis concerned with the investigation of measurement invariance.

Multiple imputation (MI) is a popular method for handling missing data. In education research, it can be challenging to use MI because the data often have a clustered structure that need to be accommodated during MI. Although much research has considered applications of MI in hierarchical data, little is known about its use in cross-classified data, in which observations are clustered in multiple higher-level units simultaneously (e.g., schools and neighborhoods, transitions from primary to secondary schools). In this article, we consider several approaches to MI for cross-classified data (CC-MI), including a novel fully conditional specification approach, a joint modeling approach, and other approaches that are based on single- and two-level MI. In this context, we clarify the conditions that CC-MI methods need to fulfill to provide a suitable treatment of missing data, and we compare the approaches both from a theoretical perspective and in a simulation study. Finally, we illustrate the use of CC-MI in real data and discuss the implications of our findings for research practice.

Job satisfaction has long been discussed as an important factor determining individual behavior at work. To what extent this relationship is also evident in the teaching profession is especially relevant given the manifold job tasks and tremendous responsibility teachers bear for the development of their students. From a theoretical perspective, teachers’ job satisfaction should be negatively related to turnover intentions and absenteeism, and positively to high-quality teacher-student interactions (i.e., emotional support, classroom management, and instructional support), enhanced student motivation, and achievement. This research synthesis provides a comprehensive overview of the relationship between teachers’ job satisfaction and these variables. A systematic literature search yielded 105 records. Random-effects meta-analyses supported the theoretically postulated relationships between teachers’ job satisfaction and their turnover intentions, absenteeism, teacher-student interactions, and students’ outcomes. Effects were significant not only for teachers’ self-reports of their professional performance, but also for external reports. On the basis of the research synthesis, we discuss theoretical, conceptual, and methodological considerations that inform future research and prospective intervention approaches.

Introduction The efficacy and effectiveness of digital interventions for depression are both well-established. However, precise effect size estimates for mediators transmitting the effects of digital interventions are not available; and integrative insights on the specific mechanisms of change in internet- and mobile-based interventions (IMIs)—as related to key features like delivery type, accompanying support and theoretical foundation—are largely pending. Objective We will conduct a systematic review and individual participant data meta-analysis (IPD-MA) evaluating the mediators associated with therapeutic change in various IMIs for depression in adults. Methods We will use three electronic databases (i.e., Embase, Medline/PubMed, PsycINFO) as well as an already established database of IPD to identify relevant published and unpublished studies. We will include (1) randomized controlled trials that examine (2) mediators of (3) guided and unguided (4) IMIs with (5) various theoretical orientations for (6) adults with (7) clinically relevant symptoms of depression (8) compared to an active or passive control condition (9) with depression symptom severity as primary outcome. Study selection, data extraction, as well as quality and risk of bias (RoB) assessment will be done independently by two reviewers. Corresponding authors of eligible primary studies will be invited to share their IPD for this meta-analytic study. In a 1-stage IPD-MA, mediation analyses (e.g., on potential mediators like self-efficacy, emotion regulation or problem solving) will be performed using a multilevel structural equation modeling approach within a random-effects framework. Indirect effects will be estimated, with multiple imputation for missing data; the overall model fit will be evaluated and statistical heterogeneity will be assessed. Furthermore, we will investigate if indirect effects are moderated by different variables on participant- (e.g., age, sex/gender, symptom severity), study- (e.g., quality, studies evaluating the temporal ordering of changes in mediators and outcomes), and intervention-level (e.g., theoretical foundation, delivery type, guidance). Discussion This systematic review and IPD-MA will generate comprehensive information on the differential strength of mediators and associated therapeutic processes in digital interventions for depression. The findings might contribute to the empirically-informed advancement of psychotherapeutic interventions, leading to more effective interventions and improved treatment outcomes in digital mental health. Besides, with our novel approach to mediation analyses with IPD-MA, we might also add to a methodological progression of evidence-synthesis in psychotherapy process research. Study registration with Open Science Framework (OSF) https://osf.io/md7pq/.

Response Surface Analysis (RSA) is gaining popularity in psychological research as a tool for investigating congruence hypotheses (e.g., consequences of self-other agreement, person-job fit, dyadic similarity). RSA involves the estimation of a nonlinear polynomial regression model and the interpretation of the resulting response surface. However, little is known about how best to conduct RSA when the underlying data are incomplete. In this article, we compare different methods for handling missing data in RSA. This includes different strategies for multiple imputation (MI) and maximum-likelihood (ML) estimation. Specifically, we consider the "just another variable" (JAV) approach to MI and ML, an approach that is in regular use in applications of RSA, and the more novel "substantive-model-compatible" (SMC) approach. In a simulation study, we evaluate the impact of these methods on focal outcomes of RSA, including the accuracy of parameter estimates, the shape of the response surface, and the testing of congruence hypotheses. Our findings suggest that the JAV approach can sometimes distort parameter estimates and conclusions about the shape of the response surface, whereas the SMC approach performs well overall. We illustrate applications of the methods in a worked example with real data and provide recommendations for their application in practice.