This paper is intended for research funders (private, philanthropic and public), policymakers, researchers and policy stakeholders with an interest in understanding how to measure and assess the wider impact of research. We take a policy analysis and synthetic approach to understanding the state of the field and the future directions of travel. This paper was originally published in 2014 and amid increasing interest in research assessment methodologies and the need to make the case for the importance of research in uncertain geopolitical climates, we hope the republication of this paper is helpful to a wider audience.

Understanding the costs of developing, writing, reviewing, and deciding on research proposals is a critical prerequisite for understanding the efficiency of funding decisions. The objective of this study was to develop estimates for cost and transaction cost – the ratio of cost to funding – for a sample of different funding schemes administered by two funders. The perceived benefits of the peer-review funding process to applicants, reviewers, and panelists were also explored. A total of 12,617 individuals were surveyed between April 2022 and June 2023, with an overall response rate of 11%. The study estimated that the cost of grant and fellowship application processes is 13% of the value of the grant and that 89% of those costs are borne by applicants. This would suggest that policies seeking to improve the efficiency of grant funding processes should consider the role of university and research institute practices alongside funder requirements and assessment processes.

A major shift in the research sector has been the increased expectation from policymakers and funders that academic research should yield some socioeconomic benefits or ‘impacts’ rather than merely new knowledge. In this paper, we explore the role that impact has in academics’ motivations and values and how impact is being integrated into academics’ core functions of research and education. We do this through in-depth interviews (n = 60) with scientists who work on the development or application of artificial intelligence (AI), broadly defined. This AI’s focus situates our participants within a strategically important, high-priority area of research for all three national contexts included in our study—Australia, the UK and the USA. Our findings reveal that the impact mission has become central to understanding the motivations and values of academics, but unevenly. We identify divergence between those who work on AI from a foundational computer science perspective and those who develop and apply AI within other scientific domains. The two groups have different understandings of key notions such as ‘impact’ and ‘applied research’, as well as different ways of integrating the impact agenda into their research and education activities. The study highlights the importance of flexible approaches to research policy and governance that are based on a deeper understanding of what motivates researchers, and that take into account academics’ educational role. Greater holistic understanding of how academic identities and practices are accommodating the impact agenda is essential to maximise synergy across activities and avoid unintended consequences.

AbstractThis chapter outlines the evolution of universities in Sweden. The chapter shows how Swedish universities have been shaped in the interplay between academic, political and economic forces and taken a specific form as an extension of political ambition, and how that model of university governance has exposed Swedish universities to both opportunities and vulnerabilities that ensue in periods of rapid ruptures. The chapter pays particular attention to the responses that have unfolded to the COVID-19 pandemic but also more generally to how universities in Sweden have balanced the orientation to global templates with that of political expectations.

This open access book explores the impact of Covid-19 students, staff, faculty and academic leaders have adapted to and dealt with the pandemic.

AbstractThis chapter outlines the background and theoretical assumptions of the book: that universities have evolved intertwined with societal development, and that the role of service to society has been ingrained in universities from their onset. The chapter introduces notions of disruption and rupture as characteristic of the university-society relationship, with the pandemic as only one of many instances in history. Finally, the chapter outlines a theory of power within universities to conceptualise how decisions are made and how the interface between universities and their surrounding society evolves.

AbstractThis chapter is composed of short recollections of events and responses during the pandemic, in particular how the experience of individual professional staff members coalesced with organisational dynamics, and how each of the contributors responded to and was affected by the pandemic, on a personal and a professional level. In this chapter, the experiences of staff form the centrepiece, and how this group adapted its ways of providing services and relating to students, faculty and leadership in so doing.

AbstractThe chapter is composed of short recollections of events and responses before and during the pandemic, in particular how the experience of individuals coalesced with the organisational, and how each of the contributors responded to and was affected by the pandemic, on a personal and a professional level. In this chapter, the experiences of leaders form the centrepiece, and how this group adapted its ways of doing management and relating to students, staff, faculty and the political systems, and how they aligned societal needs with professional development.

AbstractThis chapter outlines the evolution of universities in Australia. The chapter shows how Australian universities have been shaped in the interplay between academic, political and economic forces, and how the model of university governance has exposed them to vulnerabilities that ensue in periods of rapid ruptures in the surrounding society, especially when the globalisation of higher education is in peril. The chapter pays particular attention to the responses that have unfolded to the COVID-19 pandemic but also more generally to how universities have balanced the orientation to global templates with that of societal expectations.

AbstractThis chapter is composed of short recollections of events during the pandemic, in particular how the experience of individual students coalesced with organisational dynamics in universities, and how each of the contributors responded to and was affected by the pandemic, both on a personal and a professional level. In this chapter, the experiences of students are showcased, and how they adapted in their ways of doing studies and relating to one another as a group, and how they aligned societal needs with professional development.

AbstractOn the basis of the theoretical template of universities in society, this chapter outlines the evolution of higher education in the UK. The chapter shows the way in which British universities have been shaped in the interplay between academic, political and economic forces in recent decades, and how the adopted model of university governance has exposed them to vulnerabilities that ensue in periods of rapid ruptures in society. The chapter pays particular attention to the responses that have unfolded to the COVID-19 pandemic but also more generally to how universities have balanced the orientation to global templates with that of societal expectations.

AbstractThis chapter is composed of short recollections of events and responses during the pandemic, in particular how the experience of individual academics coalesced with the organisational dynamics of universities, and how each of the contributors responded to and was affected by the pandemic, on a personal and a professional level. The experiences of faculty form the centrepiece, and how this group adapted its ways of doing teaching and research, as well as relating to students, staff and management in so doing.

AbstractThis chapter connects the theoretical assumptions with the institutional portraits and the individual autobiographies of the book, relating these observations to the future of universities and how they may reduce vulnerabilities, in society at large, in the academic system, and within themselves.

Background: All parts of the research community have an interest in understanding research impact whether that is around the pathways to impact, processes around impact, methods for measurement, describing impact and so on. This proof of concept study explored the relationship between research funding and research impact using the case studies submitted to the UK Research Excellence Framework (REF) exercise in 2014 as a proxy for impact. Methods: The paper describes an approach to link the REF impact case studies with the underpinning research grants present in the Researchfish dataset, primarily using the publications captured in both datasets. Where possible the methodology utilised unique identifiers such as Digital Object Identifiers and PubMed ID’s, and where this was not possible the funding information within each publication was used. Results: Through this automated approach 21% of the non-redacted case studies could be linked to a specific research grant. Additional qualitative analysis was then done for unlinked REF impact case studies, which involved reading the document to identify additional information to make the linkage. This approach was taken on 100 REF impact case studies selected at random and resulted in only seven having no identifiable research grants funding associated. The linked research grants were analysed to identify characteristics that are more frequently associated with these grants, than non-linked ones. Conclusions: This analysis did point to some interesting observations such as the grant funding linked to REF impact case studies are more likely to be longer, higher financial value, have more publications and be more collaborative (amongst other characteristics). These findings should be used with caution at present and not be over interpreted, this is due to the sample size for this proof of concept study and some potential limitations on the data which were not addressed at this stage.

Clinical research infrastructure is one of the unsung heroes of the scientific response to the current COVID-19 pandemic. The extensive, long-term funding into research support structures, skilled people, and technology allowed the United Kingdom research response to move off the starting blocks at pace by utilizing pre-existing platforms. The increasing focus from funders on evaluating the outcomes and impact of research infrastructure investment requires both a reframing and progression of the current models in order to address the contribution of the underlying support infrastructure. The majority of current evaluation/outcome models focus on a "pipeline" approach using a methodology which follows the traditional research funding route with the addition of quantitative metrics. These models fail to embrace the complexity caused by the interplay of previous investment, the coalescing of project outputs from different funders, the underlying infrastructure investment, and the parallel development across different parts of the system. Research infrastructure is the underpinning foundation of a project-driven research system and requires long-term, sustained funding and capital investment to maintain scientific and technological expertise. Therefore, the short-term focus on quantitative metrics that are easy to collect and interpret and that can be assessed in a roughly 5-year funding cycle needs to be addressed. The significant level of investment in research infrastructure necessitates investment to develop bespoke methodologies that develop fit-for-purpose, longer-term/continual approach(es) to evaluation. Real-world research should reflect real-world evaluation and allow for the accrual of a narrative of value indicators that build a picture of the contribution of infrastructure to research outcomes. The linear approach is not fit for purpose, the research endeavour is a complex, twisted road, and the evaluation approach needs to embrace this complexity through the development of realist approaches and the rapidly evolving data ecosystem. This paper sets out methodological challenges and considers the need to develop bespoke methodological approaches to allow a richer assessment of impact, contribution, attribution, and evaluation of research infrastructure. This paper is the beginning of a conversation that invites the community to "take up the mantle" and tackle the complexity of real-world research translation and evaluation.

China has become an impactful science superpower, but it is asserted that its influence provides it with disproportionate benefits that some national research policies have not thus far appreciated. To create context, Web of Science data are used to analyse research collaborations between the UK and China: trends in volume of output between 1981 and 2019; citation impact; and comparative performance across research fields. UK-China collaboration increased from fewer than 100 co-authored papers before 1990, to 750 per year in 2000, 3,324 in 2010, and 16,267 papers (10.9 per cent of UK output) in 2019. UK-China collaboration is concentrated in technology-based fields: in some (e.g. telecommunication), over 30 per cent of UK papers are in collaboration with Chinese-based researchers. The paper discusses the policy consequences to the UK of this indicative dependency, arguing that exiting from such collaborations is ill advised, provided the risks, perceived or real, are mapped, managed, and mitigated.

Background Mental health remains a neglected issue on the global health policy agenda, particularly in low- and middle-income countries (LMIC), and the translation of research evidence into policy and practice is slow. The new EVITA framework was developed to improve mental health evidence uptake and policy agenda-setting in LMICs. In addition, behavioural science methods may be able to support knowledge translation to policy. Methods Using a mixed-methods study design, we applied and tested the newly developed EVITA 1.1 framework against three case studies related to South Africa at the district, national and international levels. In-depth interviews with 26 experts were conducted between August and November 2019, transcribed, coded and analysed in NVivo, using iterative categorization. The data were analysed against both the EVITA framework and the MINDSPACE framework for behavioural insights. Results In our case study comparison, we found that (1) research translation to the policy agenda occurs in a complex, fluid system which includes multiple “research clouds”, “policy spheres” and other networks; (2) mental health research policy agenda-setting is based on key individuals and intermediaries and their interrelationships; and (3) key challenges and strategies for successful research to policy agenda impact are known, but are frequently not strategically implemented, such as including all stakeholders to overcome the policy implementation gap. Our data also suggest that behavioural science methods can be strategically applied to support knowledge translation to policy agenda-setting. Conclusion We found that the EVITA framework is useful for understanding and improving mental health research policy interrelationships to support evidence uptake to the policy agenda, and that behavioural science methods are effective support mechanisms. The revised EVITA 2.0 framework therefore includes behavioural insights, for improved mental health policy agenda-setting in LMICs. More research is needed to understand whether EVITA can be applied to other LMICs and to high-income contexts.

In a referendum in March 2016, the British public voted by a margin of 52 per cent to 48 per cent to leave the European Union. Unfortunately, the referendum question provides little information on the sort of relationship Britons desire with the EU. The purpose of this study is to use stated choice experiments (CEs) to understand and quantify what aspects of the relationship between Britain and the EU are important to the public and what trade-offs they would be willing to make. The CEs are undertaken with a random probability sample of the public at two points in time, 2017 and 2018, to allow us to explore how preferences might have changed over time. We developed a series of discrete choice models ranging from Scaled Multinomial Logit models (SMNLs) to Latent Class models (LCMs) to understand and quantify public preferences and to explore how they vary across the population. The results are used to assess how positively or negatively the public value different plausible Brexit negotiation positions.