This article investigates the design and implementation of Gender Equality Plans (GEPs) across nine European universities within a university alliance, emphasising the importance of context‐specific interventions. It explores the diversity in GEPs’ content, structure, and implementation, as well as the organisation of gender equality work, revealing how national policies, institutional priorities, and socio‐cultural contexts shape these processes. Key challenges include gendered academic cultures, internal resistance, structural and organisational barriers, resource limitations, national commitment disparities, and deeply ingrained societal norms affecting gender equality. Drawing on complexity theory, this article advocates for gender‐sensitive and context‐aware frameworks to navigate the non‐linear nature of institutional interventions in academia across diverse settings. Moreover, it offers insights into the implementation of GEPs within university alliances, a context underexplored in the literature. By examining the dynamics of GEP implementation across multiple institutions within a collaborative framework, the article uniquely contributes to understanding how alliances facilitate knowledge exchange, resource sharing, and collective learning, enhancing institutional capacity and fostering synergies to more effectively address gender inequalities.

Scientific publications are important markers of achievement in academic medicine. Whilst gender disparities in scientific authorship are well documented, there is little understanding of the influence of authors’ protected characteristics on authorship positions, publication outputs and citations in biomedical and translational research. This study examined the diversity of scientific authorship by three protected characteristics, i.e., ethnicity, sex and age of authors of biomedical research publications. A retrospective longitudinal descriptive analysis of 476 unique authors’ protected characteristics, biomedical scientific publications and citations from 2012 to 2022. National Institute for Health and Care Research (NIHR) Oxford Biomedical Research Centre (BRC) in England. Peer-reviewed publications funded by the NIHR Oxford BRC during its funding round 2 (2012–2017) (henceforth BRC2) and round 3 (2017-2022) (henceforth BRC3). BRC author-level data were extracted on three self-reported protected characteristics, i.e., ethnicity, sex (male and female) and age of authors. Data were analysed by descriptive statistics, Fisher’s exact test and Pearson’s chi-squared test of independence to explore the proportions of first and last (senior) authorship positions and number of publications and citations by ethnicity, legal sex and age of authors. Authorship positions (first and last authors), number of publications and citations of publications by author ethnicity, sex and age. There were statistically significant differences in first and last authorship positions and number of publications and citations of publications by author ethnicity, sex and age. White researchers were 79% and 85% of first authors and 85% and 86% of last authors in BRC2 and BRC3, respectively. In BRC2, females were more (53%) in the first author category, while males were more (58%) in the last author category (p<0.014). In BRC3, male first (70%) and last authors (78%) were statistically significantly more than female first (30%) and last (22%) authors (p<0.001). For the total study period, 44% of first and 24% of last authors were female, which was statistically significantly less than male first (56%) and last (76%) authors (p<0.001). Authors aged 40-59 were statistically significantly more (79% and 71% of first authors and 73% and 75% of last authors in BRC2 and BRC3, respectively) than authors of other age groups (p<0.001). White authors published more publications (87%) than BAME authors (13%) in both BRC periods, and there were no differences in the overall proportions of publications by authors of either ethnicity in both BRC periods (p=0.91). In BRC2 and BRC3, male authors produced statistically significantly more publications (59% and 73%) than female authors (41% and 27%, respectively, p<0.001). Authors aged 40-59 years published more papers (64% and 69% of total papers in BRC2 and BRC3, respectively) than authors of other age groups (p<0.001). Citations of publications by white authors were statistically significantly higher (89% and 87% of total citations of BRC2 and BRC3 publications, respectively) than publications by BAME authors (p<0.001). Publications by male authors had higher citations (65% and 68% of total citations of BRC2 and BRC3 publications, respectively) than female authors’ publications (p<0.001). Citations of publications by authors aged 40-59 years were higher (56% and 72% of total citations of BRC2 and BRC3 publications, respectively) than citations of publications by authors aged less than 40 years and those aged 60 and above (p<0.001). Authors aged 60-64 years had the highest average citations per publication in both BRCs and the total period of the study. In the BRC3 period, some positive trends were observed, such as a substantial increase in the absolute number of BAME and female first and last authors and young first authors (aged up to 39 years); an increase in the number of publications by BAME, female and young authors (aged up to 39 years); and an increase in total citations of publications by BAME, female and young authors, and higher average citations of female authors’ publications (42 citations per publication) compared to male authors’ publications (33 citations per publication) in BRC3. There are statistically significant differences in diversity of authorship positions, publication output and citations by ethnicity, sex and age of authors. Positive trends include an increase in BAME, female and young authors and their publications and citations; however, under-representation of authors of BAME background, female and younger age is identified. Gaps in scientific authorship, publications and citations by author ethnicity, sex, and age could be addressed by systematic collection of data on protected characteristics of authors (researchers). In addition, tackling disparities in recruitment, career progression and research funding is imperative. This is the first study to examine diversity at author level by ethnicity, sex and age to our knowledge in three publication metrics: authorship, publications output and citations count within the context of an NIHR BRC. Most studies have examined gender individually, here we explore the influence of ethnicity, sex and age of authors on authorship type (first and last authors), publications productivity and citations count. This study provides benchmarking data on scientific authorship diversity of relevance to other NIHR funded BRCs. This study identified the challenges of benchmarking protected characteristics at individual author level because such data is not routinely available or collected. Completion of protected characteristics data is self-reported, not mandatory and therefore incomplete. However, the data does provide important benchmarking data and the feasibility of gathering such information at author level. Further work could be completed to improve the reliability and comprehensiveness of such data.

International collaboration in higher education and research is considered crucial for economic growth and development. The policy pursued for international research collaboration depends on conditions in other geographical regions and countries, such as trade interests, conflicts, security, and pandemics, hence the term geopolitics. These conditions can be expected to have a major impact on how relationships and cooperation patterns develop. The handling of these issues is discussed in this chapter in the framework of the European policy context as well as through empirical examples regarding the dependence of international recruitment of researchers in two Scandinavian countries, Denmark and Norway. The chapter points to the possible barriers current geopolitical challenges may have for the realization of European as well as national policies for internationalization in higher education. Inequalities depending on gender, career stage, country of birth and other factors have intensified during the pandemic and academic autonomy and freedom of individual researchers and scientific communities are at stake due to a geopolitical situation, which entails increased control with knowledge transfer and research collaboration across national borders. These developments require more complex, targeted safeguarding and policies to protect collaboration in higher education and research.

The evaluation of Responsible Research and Innovation (RRI) actions presents important challenges for the key stakeholders engaged in the process of RRI implementation, such as policy makers, programme managers, and researchers. While there is a considerable body of literature on the conceptualization of responsibility in research and a plethora of attempts to practice RRI, there is a need for increased attention to the monitoring and evaluation of case studies of RRI implementations in research organisations, in particular regarding their structural change effects. This paper aims to discuss a contextualised developmental framework for evaluating RRI implementation in research organisations, with a specific focus on achieving structural change through tailor-made action plans. The framework, developed through RRI evaluation work in the field of biosciences, adopts a systemic and process-oriented perspective, encompassing participatory, anticipatory, reflexive, and responsive dimensions. Concrete empirical examples from bioscience organizations are provided to illustrate how the framework relates to specific conditions, experiences, and solutions, demonstrating how conceptual insights have emerged from real-life practices and data analysis. While the framework was initially customized for the specific contexts of six bioscience research organizations, it holds potential for broader relevance and applicability in addressing challenges related to RRI design, implementation, and evaluation.

Through exploring the links between policy, evaluation, and gender, using a gender-transformative lens, this article sheds light on policy and evaluation processes of gender equality programmes, such as the political nature of evaluation and power configurations, involving resistance in the change process. Two case studies of gender equality programmes are visited to explore the links between gender policy implementation in specific contexts, and power configurations and resistance as key factors in both implementation and evaluation. We benefitted from the theoretical insights of both the feminist and the theory of change approaches, which helped us to bridge the boundaries between policy, gender scholarship, and evaluation and allowed us to reflect upon systemic factors and underlying mechanisms that are either facilitating or hindering transformation. This article highlights the benefits of applying such an evaluation framework and demonstrates why factoring in power and resistance to evaluations of gender equality programmes is necessary to explain successes or failures.

The Scandinavian countries, i.e. Denmark, Norway and Sweden are often described as European leaders within equal opportunity and diversity. However, in spite of the fact that the Scandinavian countries have implemented policies and initiated programmes to ensure gender diversity in all sectors and levels of society since the mid-1970s and beginning of the 1980s, progress is generally slow. Nonetheless, the three countries differ regarding public and political attention on the issue and hence on policies, intensity of implementation and management of gender equality policies. In this article, we focus on the representation of Scandinavian women in leadership positions in larger companies and academia, and discuss the scope and intensity of gender equality policies and their effects. Finally, we highlight key lessons learned from decades of gender equality work in the three countries.

The project Step-Change (Science Transformation in EuroPe through Citizens involvement in HeAlth, coNservation and enerGy rEsearch), funded by Horizon 2020 and launched in March 2021, is implementing five Citizen Science Initiatives (CSIs) in the fields of health, energy and environment. The CSIs will tackle the issues of wildlife conservation in Slovenia, non-alcoholic fatty liver disease in the UK, energy communities in Germany, infectious disease outbreak preparedness in Italy, and off-grid renewable energy in agriculture in Uganda. The project will bring novelty in citizen science research while contributing to broader science aspects. The overall objective of Step-Change is to explore the potential of citizen science and to formulate recommendations and instruments for better cementing this approach within research and innovation institutions as well as changing researchers’ mindsets on its value. The project, which aims to make science more socially robust, inclusive, and democratic, will ensure that research institutes make the most of what citizen science has to offer, whilst also identifying, analysing, and limiting the associated risks. Our presentation includes main features of the project, and provides an overview of the multifaceted methodology Step-Change makes use of to foster alignment of CSIs with local contexts, nurture mutual learning, and encourage self-reflection through the means of participatory evaluation exercises. Hence, the presentation provides details about the CSIs but also about the horizontal activities of the project (scoping exercise, mutual learning and training activities, participatory evaluation, stocktaking process), which have been designed to increase the relevance, inclusiveness and sustainability of the initiatives. In this respect, we also focus on the three levels of participation of the CSIs (core team members, citizen scientists and other stakeholders), to explain the mechanisms put in place to boost the level of engagement.

Citizen Science (CS) has gained increased recognition over the last two decades. This turn is occurring in strong connection with the profound transformations that have affected science over the last few decades, leading towards a new social model of science characterised by greater openness to society regarding research content actors involved, research processes, and expected societal and economic impact. CS is at the centre of this complex change dynamics as a tool that strongly sustains the shift towards the “open social model” of science supporting a new approach to the science-society relationship. However CS is rarely evaluated for its long-term structural effects on science and the science-society relationship. This article addresses this topic, having as a point of departure the ongoing EC-funded Step Change project, aimed at promoting five Citizen Science Initiatives (CSIs) in different research fields (health, energy, and environment). Under the project, an Evaluation Framework has been developed, shaping the evaluation process as a citizen science project by adopting a developmental and participatory approach. The Evaluation Framework is organised into two different but intertwined levels, one focused on the evaluation of the individual CSI (analytical level) and a second aimed at identifying recurrent patterns of obstacles, facilitating factors, or a mix of them (neutral situations) across the CSIs (cross-cutting level). While the analytical level is intended as a service to better implement the CSIs, the cross-cutting level is intended as a research process to generate new knowledge on how CS could serve as a tool for a better anchorage of science into society.

Gender-based violence and sexual harassment (GBVH) by and towards academics and students has been under-theorised at an organisational level in higher education institutions (HEIs). The methodology involves a critical review of the literature on GBVH and organizational responses to it, locating it in the context of an analysis of organizational power. The theoretical perspective involves a focus on power and workplace bullying. It identifies three power-related characteristics of academic environments which it is suggested facilitate GBVH: their male-dominant hierarchical character; their neoliberal managerialist ethos and gender/intersectional incompetent leadership which perpetuates male entitlement and toxic masculinities. These characteristics also inhibit tackling GBVH by depicting it as an individual problem, encouraging informal coping and militating against the prosecution of perpetrators. Initiating a discussion and action at organizational and state levels about GBVH as a power-related phenomenon, challenging the dominant neo-liberal ethos and the hierarchical character of HEIs, as well as reducing their male dominance and increasing the gender competence of those in positions of power are seen as initial steps in tackling the problem.

While women remain underrepresented in healthcare leadership, an evidence base on organisational interventions that can help to accelerate their advancement to leadership positions is limited and scattered across different sectors. In an article published in EClinicalMedicine, Helena Teede and colleagues contribute to building such an evidence base by identifying and synthesising organisational interventions that have been shown to measurably advance women in leadership [[1]Mousa M. Boyle J. Skouteris H. et al.Advancing women in healthcare leadership: a systematic review and meta-synthesis of multi-sector evidence on organisational interventions.EClinicalMedicine. 2021; 39101084https://doi.org/10.1016/j.eclinm.2021.101084Summary Full Text Full Text PDF PubMed Scopus (5) Google Scholar]. Teede and colleagues systematically searched the relevant multi-disciplinary databases and identified 91 eligible studies across academia, health, government, sports, hospitality, finance, and information technology sectors, which were published in English in peer-reviewed journals between January 2000 and March 2021. Amongst these studies, there were more studies from academic medicine and healthcare than from any other sector. The authors narratively synthesised the findings from these studies using a meta-synthesis approach with a view to generating insights on the processes that enable women to advance in healthcare leadership. The multi-sector scope of the review and its meta-synthesis approach make a unique and valuable addition to other recent reviews on gender equity in healthcare [[2]Laver K.E. Prichard I.J. Cations M. et al.A systematic review of interventions to support the careers of women in academic medicine and other disciplines.BMJ Open. 2018; 8e020380https://doi.org/10.1136/bmjopen-2017-020380Crossref PubMed Scopus (50) Google Scholar,[3]Tricco A.C. Bourgeault I. Moore A. et al.Advancing gender equity in medicine.Can Med Assoc J. 2021; 193: E244-EE50https://doi.org/10.1503/cmaj.200951Crossref Scopus (11) Google Scholar]. The results of the review have the potential to inform organisational strategies, policies, and practices. Namely, the results indicate that organisational leadership, commitment, and accountability are associated with measurable improvements in women's advancement. The results identify five categories of potentially effective organisational interventions: (i) organisational processes, (ii) awareness and engagement, (iii) mentoring and networking, (iv) leadership development, and (v) support tools. Importantly, the authors acknowledge the shortcomings of isolated interventions and argue for a multifaceted approach including a range of interventions at multiple organisational levels. Moreover, the article poses important implications for the development of the field of diversity interventions. One important implication is the quality of evidence on diversity interventions. The article shows that currently there is a paucity of robust quantitative studies, a varied body of qualitative studies, and a lack of standardised outcome measures across both quantitative and qualitative studies. This is in line with previous research showing that even in the case of mentoring, which is arguably the most common diversity and career advancement intervention, it is impossible to ascertain its effectiveness in reducing gender inequalities due to a lack of standardised approaches and weak evaluation designs [[4]House A. Dracup N. Burkinshaw P. et al.Mentoring as an intervention to promote gender equality in academic medicine: a systematic review.BMJ Open. 2021; 11e040355https://doi.org/10.1136/bmjopen-2020-040355Crossref PubMed Scopus (5) Google Scholar]. Given that research funders, professional associations, and research organisations across the globe commit significant resources and efforts to diversity interventions, there is a need to develop standardised and more rigorous approaches to designing and evaluating diversity interventions. For example, government funding agencies and professional associations in the United Kingdom, Ireland, Australia, the United States, and Canada have been instrumental in the wide-spread adoption by higher education and research institutions of gender equality action plans based on the standardised Athena Swan framework [[5]Ovseiko P.V. Taylor M. Gilligan R.E. et al.Effect of Athena SWAN funding incentives on women's research leadership.BMJ. 2020; 371: m3975https://doi.org/10.1136/bmj.m3975Crossref PubMed Scopus (9) Google Scholar]. A new European gender equality strategy has made provisions for the European flagship research funding programme to require applicants to have in place gender equality action plans [[6]Kalpazidou Schmidt E. Ovseiko P.V. Link Horizon Europe funding to real steps to gender equality.Nature. 2020; 584: 525https://doi.org/10.1038/d41586-020-02430-1Crossref PubMed Scopus (5) Google Scholar]. The Australian Academy of Science has identified establishing a consistent national evaluation framework across all gender equity initiatives in science, technology, engineering and mathematics as one of the major strategic opportunities of the current decade [[7]Australian Academy of ScienceWomen in STEM decadal plan. Australian Academy of Science, 2019https://www.science.org.au/womeninSTEMplanGoogle Scholar]. Another important implication for the development of the field of diversity interventions posed by the article is the complexity of organisational interventions, which requires continuously adapting interventions to the local contexts and constantly emerging conditions [[8]Kalpazidou Schmidt E. Ovseiko P.V. Henderson L.R. et al.Understanding the Athena SWAN award scheme for gender equality as a complex social intervention in a complex system: analysis of Silver award action plans in a comparative European perspective.Health Res Policy Syst. 2020; 18https://doi.org/10.1186/s12961-020-0527-xCrossref PubMed Scopus (26) Google Scholar,[9]Kalpazidou Schmidt E. Ovseiko P.V. Acknowledging complexity in evaluation of gender equality interventions.EClinicalMedicine. 2020; 28https://doi.org/10.1016/j.eclinm.2020.100623Summary Full Text Full Text PDF PubMed Scopus (1) Google Scholar]. It follows that the effectiveness of interventions will also depend on the capabilities of the local staff in adapting and implementing interventions taking into account their local contexts and emerging conditions. As such, in parallel with standardising and strengthening the design and evaluation of diversity interventions, there is a need to develop implementation science approaches and tools to support the effective implementation of diversity interventions. Yet another important implication for the development of the field of diversity interventions posed by the article is the salience of sector and organisational cultural factors. The authors rightly point out that there is a need for future research to gain greater insight into organisational cultures in the strongly hierarchical healthcare sector. While the hierarchical cultures in the health sector can present barriers to women's career advancement, they can also help to ensure the uptake and effective implementation of gender equity interventions when healthcare leaders commit themselves to advancing gender equity [[5]Ovseiko P.V. Taylor M. Gilligan R.E. et al.Effect of Athena SWAN funding incentives on women's research leadership.BMJ. 2020; 371: m3975https://doi.org/10.1136/bmj.m3975Crossref PubMed Scopus (9) Google Scholar,[10]Ovseiko P.V. Pololi L.H. Edmunds L.D. et al.Creating a more supportive and inclusive university culture: a mixed-methods interdisciplinary comparative analysis of medical and social sciences at the University of Oxford.Interdiscip. Sci Rev. 2019; 44: 166-191https://doi.org/10.1080/03080188.2019.1603880Crossref Scopus (20) Google Scholar]. The fact that the current review includes more studies from academic medicine and healthcare than from any other sector suggests that the healthcare sector is leading multi-sector efforts in evidence-based gender equality interventions. The evidence synthesis presented in the current article makes a valuable contribution to help accelerate such efforts further. PVO declares an advisory role with the Advance HE Athena Swan Governance Committee. EKS declares no conflicts of interest. PVO is supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre, Grant BRC-1215–20008 to the Oxford University Hospitals NHS Foundation Trust and the University of Oxford, and by the European Union's Horizon 2020 research and innovation programme under grant agreement No. 872396 . EKS is supported by the European Union´s Horizon 2020 research and innovation programme under Grant agreements Nos. 872146, 101006386 , and by the Norwegian Research Council through the Gender, Citizenship and Academic Power (GAP) project. Advancing women in healthcare leadership: A systematic review and meta-synthesis of multi-sector evidence on organisational interventionsThis review provides an evidence base on organisational interventions for advancing women in leadership across diverse settings, with lessons for healthcare. It transcends the focus on the individual to target organisational change, capturing measurable change across intervention categories. This work directly informs a national initiative with international links, to enable women to achieve their career goals in healthcare and moves beyond the focus on barriers to solutions. Full-Text PDF Open Access

Evaluation cultures and evaluation capacity building vary greatly across the European Union. Western European countries, such as Austria, Germany, Denmark and Sweden, have been termed as leading countries in the evaluation as they have built up well-established evaluation cultures and carry out systematic evaluations of programmes and institutions. In contrast, in Central and Eastern European (CEE) countries, efforts continue to establish evaluation practices and further develop the current evaluation culture. In Hungary, for example, an established research and innovation evaluation practice does not exist, not one specifically considering gender equality in research and innovation evaluations with the exception of research and innovation programmes financed by the EU Structural Funds. Based on the results of a Horizon 2020 project, we apply a context-sensitive evaluation concept in Hungary that enables program owners and evaluators to develop a tailor-made design and impact model for their planned or ongoing gender equality interventions. The development of this evaluation was based on a thorough analysis of the literature and 19 case studies, building on documentary analysis and semi-structured interviews. The article shows that this evaluation approach is applicable also in countries with a certain catch-up demand of the existing overall evaluation culture. The special feature of the presented evaluation approach is, on the one hand, that the evaluation is context-sensitive. On the other hand, this approach makes it possible not only to depict effects on gender equality itself, but also to anticipate effects on research and innovation. Such effects can, for example, be a stronger orientation of research towards societal needs, which makes it particularly interesting for private companies.

Experiences from individualized gender equality funding programs, as the ones used in Denmark, demonstrate that one-off policy interventions, although a small step in the right direction, cannot stand alone in the fight against gender imbalances in academia. Closing the gender gap is a complex, multi-level undertaking that needs constant rethinking of policies and the dedication of adequate financial resources. The need of rethinking policy is in particular urgent during Covid-19, which has further amplified imbalances due to a drop in the productivity of women researchers. Funding bodies should therefore reconsider traditional approaches heavily rewarding publications in the distribution of research funds. They ought to respond to the gendered impact of the pandemic by engaging institutions in structural and cultural change, setting up requirements for institutions to have achieved a certain level of gender equality outcomes, and thus link institutional progress to research funding.

Background Given the complex mix of structural, cultural and institutional factors that produce barriers for women in science, an equally complex intervention is required to understand and address them. The Athena SWAN Award Scheme for Gender Equality has become a widespread means to address barriers for women’s advancement and leadership in the United Kingdom, Ireland, Australia, the United States of America and Canada, while the European Commission is exploring the introduction of a similar award scheme across Europe. Methods This study analyses the design and implementation of 16 departmental Athena SWAN Silver Action Plans in Medical Sciences at one of the world’s leading universities in Oxford, United Kingdom. Data pertaining to the design and implementation of gender equality interventions were extracted from the action plans, analysed thematically, coded using categories from the 2015 Athena SWAN Charter Awards Handbook and synthesised against a typology of gender equality interventions in the European Research Area. The results were further analysed against the complexity research literature framework, where research organisations are perceived as dynamic systems that adapt, interact and co-evolve with other systems. Results Athena SWAN is a complex contextually embedded system of action planning within the context of universities. It depends on a multitude of contextual variables that relate in complex, non-linear ways and dynamically adapt to constantly moving targets and new emergent conditions. Athena SWAN Silver Action Plans conform to the key considerations of complexity – (1) multiple actions and areas of intervention with a focus on the complex system being embedded in local dynamics, (2) the non-linearity of interventions and the constantly emerging conditions, and (3) impact in terms of contribution to change, improved conditions to foster change and the increased probability that change can occur. Conclusions To enact effective sustainable structural and cultural change for gender equality, it is necessary to acknowledge and operationalise complexity as a frame of reference. Athena SWAN is the single most comprehensive and systemic gender equality scheme in Europe. It can be further strengthened by promoting the integration of sex and gender analysis in research and education. Gender equality policies in the wider European Research Area can benefit from exploring Athena SWAN’s contextually embedded systemic approach to dynamic action planning and inclusive focus on all genders and categories of staff and students.

In this article, we discuss the development of a conceptual evaluation framework to design and assess gender equality interventions and their effects in research and innovation. The conceptual framework presented herewith embraces the complexity, gender-sensitive and theory-based evaluation approaches ensuring that design and evaluation of gender equality interventions consider the complex systems that constitute the context in which the interventions operate. The evaluation framework offers a non-linear concept, where the notion of contribution - not attribution - to achieve impact is central to the integration of team, organizational and system factors in policy design and evaluation. The paper opens the "black box" to address the question of how and why a policy intervention works and in which context and discusses a systematic process on how to approach the interwoven linkages between input, implementation and effects in gender equality interventions in research and innovation, accounting for context sensitivity and methodological pluralism. The evaluation framework may serve as reference for researchers, evaluators, policymakers and other stakeholders in designing and assessing gender equality interventions, and in further developing their evidence, and theoretical and methodological base.

Despite the fact that the topic of “women in research and innovation” has been on the agenda for decades and numerous measures have been implemented at both national and supranational levels to improve gender equality in research and innovation systems, it is still unclear which measures and under which conditions these measures are most effective. Even less research has been carried out on the effects of better representation of women in terms of research and innovation. This paper is based on the application of an innovative evaluation framework, which encompasses complexity and theory of change approaches and aims at exploring the link between interventions and their subsequent effects to two case studies. We discuss two major German flagship programmes aiming at increasing the participation of female researchers in the science system, the “Women Professorship Programme” and the “Pact for Research and Innovation”. Through the two programmes, we tested and validated the evaluation framework and its indicators. As part of the validation process, a theory of change has been developed for each of the programmes. The theory-based evaluation approach helped not only to identify gender equality impacts but also broader effects on research and innovation that might have otherwise remained undetected. We studied the effects of the two programmes on the number of women in leadership positions and analysed whether an increase in the proportion of women leaders influences publication patterns. Although linear linkages are challenging to establish due to the complexity of the process, the findings suggest that the flagship programmes have contributed not only to higher shares of women researchers but also to improved female publication and citation rates. There are clear benefits for Germany in terms of scientific results from the increased proportion of female researchers in research and innovation.

A collection of evidence on gender equality published in EClinicalMedicine discusses the systemic nature of restrictive gender norms in science and medicine, and calls attention to the importance of institutional interventions to overcome restrictive gender norms[[1]Darmstadt G.L. Gender equality: framing a special collection of evidence for all.EClinicalMedicine. 2020; 20https://doi.org/10.1016/j.eclinm.2020.100307Summary Full Text Full Text PDF Scopus (1) Google Scholar]. Inequality is deep-rooted in science and medicine while evaluation of interventions show a very slow progress and often unintended consequences [[2]Moughalian C. Täuber S. When gender equality initiatives risk doing more harm than good.EClinicalMedicine. 2020; 22https://doi.org/10.1016/j.eclinm.2020.100330Summary Full Text Full Text PDF PubMed Scopus (1) Google Scholar]. For example, women often undertake a disproportionate amount of gender equality work, institutional gender equality plans can become box-ticking exercises, men can feel discriminated against, and gender can take prevalence over race and class [[3]Kalpazidou Schmidt E. Ovseiko P.V. Henderson L.R. Kiparoglou V. Understanding the Athena SWAN award scheme for gender equality as a complex social intervention in a complex system: analysis of Silver award action plans in a comparative European perspective.Health Res Policy Syst. 2020; 18https://doi.org/10.1186/s12961-020-0527-xCrossref PubMed Scopus (19) Google Scholar]. While the analysis of the reasons for inequality and the need for policy interventions is well established, evaluation of gender equality interventions is challenging due to the complex nature of gender norms and many interacting factors. In particular, recent studies of gender equality interventions draw attention to the difficulties in attributing effect and in evaluating their overall impact [4Kalpazidou Schmidt E. Cacace M. Addressing gender inequality in science: the multifaceted challenge of assessing impact.Research Evaluation. 2017; 26: 102-114https://doi.org/10.1093/reseval/rvx003Crossref Scopus (17) Google Scholar, 5Rosser S.V. Barnard S. Carnes M. Munir F. Athena SWAN and ADVANCE: effectiveness and lessons learned.The Lancet. 2019; 393: 604-608https://doi.org/10.1016/S0140-6736(18)33213-6Summary Full Text Full Text PDF PubMed Scopus (26) Google Scholar, 6Kalpazidou Schmidt E. Graversen E.K. Developing a conceptual evaluation framework for gender equality interventions in research and innovation.Eval Program Plann. 2020; 79101750https://doi.org/10.1016/j.evalprogplan.2019Crossref PubMed Scopus (9) Google Scholar]. We concur that evaluating the impact of such complex interventions is indeed problematic when impact evaluation is solely predicated on attribution and linear causality. Here, we argue that the solution lies in acknowledging and operationalising complexity as a frame of reference and call for a paradigm shift in evaluating gender equality interventions. Key considerations of complexity in gender equality interventions involve [[3]Kalpazidou Schmidt E. Ovseiko P.V. Henderson L.R. Kiparoglou V. Understanding the Athena SWAN award scheme for gender equality as a complex social intervention in a complex system: analysis of Silver award action plans in a comparative European perspective.Health Res Policy Syst. 2020; 18https://doi.org/10.1186/s12961-020-0527-xCrossref PubMed Scopus (19) Google Scholar,[4]Kalpazidou Schmidt E. Cacace M. Addressing gender inequality in science: the multifaceted challenge of assessing impact.Research Evaluation. 2017; 26: 102-114https://doi.org/10.1093/reseval/rvx003Crossref Scopus (17) Google Scholar,[7]Kalpazidou Schmidt E. Cacace M. Setting up a dynamic framework to activate gender equality structural transformation in research organizations.Sci Public Policy. 2019; 46: 321-338https://doi.org/10.1093/scipol/scy059Crossref Scopus (14) Google Scholar]. First, multiple actions and areas of intervention. For example, Athena SWAN gender equality action plans in the UK have on average more than 30 actions addressing five major areas (organisation and culture, career development, self-assessment and monitoring, key career transition points, flexible working and career breaks)[[3]Kalpazidou Schmidt E. Ovseiko P.V. Henderson L.R. Kiparoglou V. Understanding the Athena SWAN award scheme for gender equality as a complex social intervention in a complex system: analysis of Silver award action plans in a comparative European perspective.Health Res Policy Syst. 2020; 18https://doi.org/10.1186/s12961-020-0527-xCrossref PubMed Scopus (19) Google Scholar]. In the USA, the ADVANCE program provides institutions with competitive grants, which commonly support multiple interventions including gender-disaggregated data collection, mentoring schemes, work-life balance policies, and guidance on enhancing faculty careers for women in STEM [[5]Rosser S.V. Barnard S. Carnes M. Munir F. Athena SWAN and ADVANCE: effectiveness and lessons learned.The Lancet. 2019; 393: 604-608https://doi.org/10.1016/S0140-6736(18)33213-6Summary Full Text Full Text PDF PubMed Scopus (26) Google Scholar]. Second, focus on the local dynamics. Interventions are tailored to the local contexts of specific institutions in order to disrupt local self-organisation processes maintaining gender norms – one size does not fit all. Third, non-linear nature of interventions. Due to the high number of variables involved in gender equality interventions and their constantly emergent character, effects cannot be directly attributed to interventions. Fourth, dynamic adaptation to constantly emerging conditions. A continuous monitoring and adaptation of interventions in response to implementation feedback, new emerging conditions, unintended consequences, and changes in the wider social, economic, and political context is crucial to achieve structural and cultural change. Fifth, probabilistic nature of change. Impact of gender equality interventions is expected in terms of contribution to change, improved conditions to foster change, and working to increase the probability that change can occur. Such considerations of complexity emerge from a number of studies published in the recent collection on gender equity in EClinicalMedicine. For example, a national survey of Canadian medical students' experiences of sexual harassment demonstrates that when official policies regarding sexual harassment in medical education fail to disrupt societal gender norms, such policies are not only ineffective, but can also inadvertently cause harm to victims [[8]Phillips S.P. Webber J. Imbeau S. et al.Sexual harassment of canadian medical students: a national survey.EClinicalMedicine. 2019; 7: 15-20https://doi.org/10.1016/j.eclinm.2019.01.008Summary Full Text Full Text PDF PubMed Scopus (16) Google Scholar]. Several association studies also highlight the complex causality and non-linear impact of gender equality interventions. For example, a nationally-representative study from India suggests that current policy efforts focused on affecting sex ratio imbalance are unlikely to succeed without challenging social norms regarding son preference and reduced care for infant girls [[9]Raj A. Johns N.E. McDougal L. et al.Associations between sex composition of older siblings and infant mortality in India from 1992 to 2016.EClinicalMedicine. 2019; 14: 14-22https://doi.org/10.1016/j.eclinm.2019.08.016Summary Full Text Full Text PDF PubMed Scopus (3) Google Scholar]. A study based on data from 97 countries shows that greater gender parity in education and work is associated with better health outcomes not only for females, but also for males [[10]Gadoth A. Heymann J.. Gender parity at scale: examining correlations of country-level female participation in education and work with measures of men's and women's survival.EClinicalMedicine. 2020; 20100299https://doi.org/10.1016/j.eclinm.2020Summary Full Text Full Text PDF PubMed Scopus (4) Google Scholar]. Overall, we argue for acknowledging and operationalising complexity as a frame of reference in gender equality interventions and for a paradigm shift towards impact evaluation models open to context-sensitivity and emergent causality, non-linearity, and a probabilistic nature of change. EKS and PVO co-wrote this commentary. EKS and PVO have nothing to disclose. EKS was supported by the Aarhus University Research Foundation. PVO is supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre, grant BRC-1215–20008 to the Oxford University Hospitals NHS Foundation Trust.

Gender Equality (GE) is on the top of the EU agenda, as recently confirmed in the new GE strategy, "A Union of Equality: Gender Equality Strategy 2020-2025" (European Commission 2020).GE is also laid down in Article 8 of the Treaty on the Functioning of the European Union: "In all its activities, the Union shall aim to eliminate inequalities, and to promote equality, between men and women".The scientific knowledge about the underrepresentation of women within research and innovation (R&I) systems and the most important hindrances are comprehensive and a EVANTHIA KALPAZIDOU