Introduction This tutorial shows how to measure open access by funder over time using open research information resources provided by the ORION community on Google BigQuery. We compare NIH- and European Commission-supported biomedical research literature with a focus on open access, including an estimation of articles enabled by transformative agreements.

The rise of open research information resources is transforming the way we track, analyse and study research systems. Increasingly, sources like OpenAIRE, OpenAlex, Crossref, DataCite, ORCID, ROR and others are being used as the basis for making decisions, designing interventions and understanding progress in the science system.

This paper examines the role of open research information in advancing Open Science in Ukraine, with particular attention to alignment with the Barcelona Declaration on Open Research Information. It argues that open, interoperable, and reusable research metadata constitutes a critical enabling layer for effective science governance, policy monitoring, and international collaboration. The study analyzes the current state of Open Science implementation in Ukraine, including the National Plan for Open Science, and highlights the contribution of key infrastructures such as ORCID, the Open Ukrainian Scientific Content Initiative (OUCI), and the Ukrainian Research Information System (URIS). Drawing on both international and national literature, the paper identifies major challenges, including fragmented research information systems, limited institutional capacity, and insufficient regulatory frameworks. At the same time, it demonstrates that Ukrainian institutions, particularly academic libraries, play an increasingly important role in promoting transparency, data stewardship, and interoperability. The paper concludes that engagement with the principles of the Barcelona Declaration provides a strategic pathway for strengthening Ukraine’s research ecosystem, enhancing its resilience, and facilitating its integration into the European Research Area.

This is a cross-post from Upstream supported by FORCE 11. https://doi.org/10.54900/2pnyq-nhx95 The rise of open research information resources is transforming the way we track, analyse and study research systems. Increasingly, sources like OpenAIRE, OpenAlex, Crossref, DataCite, ORCID, ROR and others are being used as the basis for making decisions, designing interventions and understanding progress in the science system.

This is a set of files with description of 237 open science practices. It also has formatted cards for all practices that can be printed and used in a variety of settings.

There is much to celebrate when it comes to recent developments in scholarly communication and research assessment. The scholarly communication landscape shows many promising initiatives, including a growing use of preprint services, open access repositories, and open peer review platforms, as well as an increasing interest in more equitable scholarly publishing models such as diamond open access and Subscribe to Open.

(Watch the VIDEO.) Research information, or scholarly metadata, is important for decision making around strategic priorities, distribution of resources, and evaluation of researchers and institutions. It is also used by funders, institutions and governments to assess the effect of policies, and by researchers and societal stakeholders to find and assess research results. The value of openly available research information is increasingly recognized, as it supports fair research assessment and makes it possible for everyone to find and assess relevant research. Publishers play an important role in this context, as they are an important source of scholarly metadata - including bibliographic metadata for the research articles, books and book chapters, and other research outputs they publish. Many publishers make important metadata, including abstracts, authors and affiliations, references and funding information available via Crossref, including persistent identifiers (ORCID, ROR and funder and grant IDs). However, for many research articles, this metadata is still only available through closed, proprietary systems, if at all. This panel brings together a number of approaches to support publishers to make metadata for the articles and other outputs they publish openly available. The panel will present and discuss these approaches, including criteria for success, potential challenges and context-specific considerations. Approaches represented on the panel vary from inclusion of open metadata in publisher negotiations, supporting under-resourced publishers, and the SCOAP3 open science mechanism that financially rewards publishers for open science practices, including open metadata. In addition, removing barriers in publisher workflows, from submission systems to metadata depositing, will be discussed. Given the diversity of publishers in the scholarly communication landscape, from large commercial publishers to scholarly societies and institutional publishers of diamond journals, challenges and solutions for the provision of open scholarly metadata will vary. The panel will invite audience perspectives on the question how publishers can make more and better metadata available as open research information.

Several preprint review communities have emerged in the life sciences in recent years. They have adopted different approaches to create and share preprint review information. New preprint review initiatives need to understand the benefits and challenges of the current technical solutions to store and transfer review metadata. This requires a clear overview of the various workflows that exist in this space. This publication maps out the preprint review metadata transfer ecosystem within the life sciences. Here we describe six illustrative workflows, as well as protocols, schemas and frameworks used for preprint review metadata transfer. For each workflow we highlight benefits and challenges faced by the preprint review services. Finally, we offer a summary of factors to consider when selecting a metadata workflow. This includes technical needs, costs, maintenance and sustainability, discoverability, and linking between preprints and reviews. We recommend that new preprint review groups adopt one of the existing workflows and encourage them to participate in community efforts to evolve and improve current standards. We believe this will support interoperability and bolster the development of new tools for preprint review.

Abstracts are increasingly important given the rise of large language models (LLM) and generative AI. While OpenAlex provides a source of open abstracts in addition to Crossref, the takedown of abstracts from OpenAlex by two major publishers points to an increased commodification of abstracts. The case for open abstracts Launched in 2020, the Initiative for Open Abstracts (I4OA) advocates and promotes the unrestricted

In academia, assessment is often narrow in its focus on research productivity, its application of a limited number of standardised metrics and its summative approach aimed at selection. This approach, corresponding to an exclusive, subject-oriented concept of talent management, can be thought of as at odds with a broader view of the role of academic institutions as accelerating and improving science and scholarship and its societal impact. In recent years, open science practices as well as research integrity issues have increased awareness of the need for a more inclusive approach to assessment and talent management in academia, broadening assessment to reward the full spectrum of academic activities and, within that spectrum, deepening assessment by critically reflecting on the processes and indicators involved (both qualitative and quantitative). In terms of talent management, this would mean a move from research-focused assessment to assessment including all academic activities (including education, professional performance and leadership), a shift from focus on the individual to a focus on collaboration in teams (recognising contributions of both academic and support staff), increased attention for formative assessment and greater agency for those being evaluated, as well as around the data, tools and platforms used in assessment. Together, this represents a more inclusive, subject-oriented approach to talent management. Implementation of such changes requires involvement from university management, human resource management and academic and support staff at all career levels, and universities would benefit from participation in mutual learning initiatives currently taking shape in various regions of the world. Keywords Academic culture Agency Culture change Evaluation process Impact Metrics Qualitative indicators Quantitative indicators Research assessment Citation Kramer, B. and Bosman, J. (2024), "Recognition and Rewards in Academia – Recent Trends in Assessment", Thunnissen, M. and Boselie, P. (Ed.) Talent Management in Higher Education (Talent Management), Emerald Publishing Limited, Leeds, pp. 55-75. https://doi.org/10.1108/978-1-80262-685-820241004 Publisher: Emerald Publishing Limited Copyright © 2024 Bianca Kramer and Jeroen Bosman License Published by Emerald Publishing Limited. These works are published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of these works (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode. The Role of Assessment in Shaping Academia For institutions that include acceleration and improvement of science and scholarship as well as their societal impact in their mission, it is important to use assessment of academic activities and their outcomes that align with those missions. Assessment of academic activities, both within and outside universities, determines how research budgets are allocated and who is hired, promoted and given tenure. It also plays a role in how the university is viewed as a partner for international collaborations, and whether potential new students and employees view the university as a desirable place to study and work. Therefore, universities are strategic in how they want to be perceived compared to other institutions, both nationally and internationally. This has a direct link with talent management – the way a university thinks about talent and how to best attract and sustain it. In literature on talent management, a distinction is made between talent conceived of as subject (with a focus on people as 'talents') and talent viewed as object (where 'talents' are characteristics of people, such as abilities, knowledge and/or competencies). In addition, a distinction is made between exclusive and inclusive talent management, with exclusive talent management focusing on selection of people or characteristics at the exclusion of others, and inclusive talent management as more broadly considering the need for multiple qualities to support the organisation's overall objective (Thunnissen et al., 2013). How a university approaches talent management has a direct relationship with how assessment is taking place. This raises questions as to how to shape assessment of academic activities to promote the academic culture that research organisations aspire to, internally as well as for the higher education system as a whole. Choices in assessment of academic activities (whom to assess, what to assess and how to assess) have the potential to shape both the institution and the wider system of higher education and research, and care should be taken to align assessment practices with the core values of the institution and the system as a whole. In essence, then, the question becomes: what kind of institution do universities want to be? From that, choices in assessment practices follow. Importantly, this way of thinking also provides a key to change when current assessment practices do not support these core values – when certain essential academic activities and roles are undervalued compared to others, when a focus on competition fosters a culture of individualism rather than teamwork and collaboration and when success is defined by narrow measures of quality and impact rather than reflect the multiple qualities of academia (Advisory Council for Science, Technology and Innovation (AWTI), 2023) that together provide true relevance for science and society. The previous chapters of this book have described how the academic landscape has changed and how open science is altering academic tasks, systems and structures. In this chapter, we will discuss what this means for assessment, by looking at current developments in the Netherlands and internationally. What choices can be made by a university in how hiring, promotion and tenure decisions are made, both for academic and non-academic staff? What are the issues with commonly used metrics for academic success, and what alternative approaches are being proposed? This chapter will first discuss the concept of both broadening and deepening assessment: rewarding the full spectrum of academic activities and, within that spectrum, critically reflecting on the process of quality and impact assessment. This will be followed by a closer look at the role of qualitative and quantitative assessment and appropriate use of indicators in both. Next, the relation that open science and research integrity play in assessment will be discussed, as well as the importance of equity and open infrastructure, with a special look at university rankings. Finally, some examples will be given of how changes in assessment practices are implemented at different academic institutions in the Netherlands, and a number of actionable international developments will be highlighted that could provide a springboard for further action. Broadening and Deepening Assessment For assessment practices to optimally support the role of universities to accelerate and improve science and scholarship and its societal impact, two aspects are important. First, they should reward the full spectrum of academic activities and not focus primarily on research. Second, they should reward practices that improve the quality, relevance and impact of academic activities, using appropriate indicators and processes. These two aspects can be conceived of as 'broadening' and 'deepening' assessment. Traditional assessment is often relatively narrow, with its limitation to research and within that to (journal) publications. It is also often relatively shallow, with the orientation at measurable output, the importance of quantity and the use of a small number of standardised metrics. Aubert Bonn and Bouter (2023) describe how metrics use in assessment developed from mere quantitative measurement to impact measurement through citation counts and journal impact factors (JIFs) but also how both provided incentives that could harm research. In addition, traditional assessment often uses a comparative–summative approach aimed at selection (Aguinis et al., 2020; Kallio et al., 2017). Finally, it often employs 'excellence' as its central tenet (Moore et al., 2017). The concept of 'excellence', while difficult to define, is used by many institutions and underpins approaches that are highly selective and often based on proxies for quality, such as journal metrics or lists of approved journals. Obviously, the exact set of criteria used differs between institutions and between the various assessment contexts such as hiring, tenure and promotion, grant allocation, prizes, etc. (see, e.g., McKiernan et al., 2019; Moher et al., 2018). In terms of talent management, this approach to assessment corresponds to an exclusive, subject-oriented approach, Thunnissen et al. (2021) focused on individual performance appraisal (Boselie, 2014). In contrast, broadening and widening assessment can be seen as moving towards a more inclusive, object-oriented approach. Broadening Assessment The concept of broadening assessment of academic activities means acknowledging that for a university to meet the expectations set upon it, more is needed than high-quality and relevant research. Education, for one, forms a large part of what a university is, and to do it well, it should be recognised and rewarded as an academic activity at par with doing research, with enough time, resources and recognition allocated to it. Many academic activities also fall under 'professional performance', be it clinical work at university hospitals or university veterinary hospitals, serving on governing or advisory boards of professional societies or associations, editorial work for scholarly journals and books, to name but a few activities. Another important area is leadership: time invested in managing a research group, fostering an open and inclusive research culture and mentoring trainees are important activities, relevant in all stages of an academic career. Two other important aspects of leadership are taking the lead and responsibility in innovation and improvement of processes and services and personal leadership: self-reflection in order to perform well. It should be apparent that these activities require dedicated time and skills and thus need to be recognised and rewarded as valuable activities on their own. When academics are primarily valued for their research activities and outcomes, but at the same time are expected to carry out these other tasks as well, this can result in overburdening people who are expected to do it all, or creating 'second-class citizens' within academia, e.g., when teaching is performed by people on temporary contracts who have less favourable career opportunities within academia. A corollary of the above is that no single person can or should be expected to excel at all academic activities or, in other words, be the elusive 'sheep with five legs'. Rather, success in academia is a team effort, and recognising this in assessment opens the door for more diverse career paths that are considered equally valuable. It also makes it easier to value contributions by support staff as bona fide academic activities – including, but not limited to, activities of lab technicians, data stewards, research software engineers, librarians who often work closely together with research groups and contribute to research, teaching and professional performance. Recognising these contributions also fits with assessment at team level, where the focus is on the functioning of the team as a whole and the contributions of all team members. Ultimately, research, education and professional performance, supported through leadership and team science, result in impact. This can be either scientific or scholarly impact (e.g., contributions to theory, methodology or results leading to practical applications) or societal impact (e.g., practical applications, contributions to societal discourse, public–private partnerships). Here, too, broadening the concept of impact in assessment is important to value diverse academic activities more equitably, rather than focus on a narrow sense of research impact as most valuable, or consider 'impact' as societal impact only, separate from impact from research or education. In the Netherlands, examples of the idea of broadening assessment can be found in the joint position paper 'Room for everyone's talent' (VSNU, 2019) and the implementation of the ambitions expressed therein at each individual university. Utrecht University has operationalised the concept of TRIPLE (Fig. 4.1) with Research, Education and Professional Performance supported and enabled by Team Science and Leadership as the scaffolding for broadening assessment (Utrecht University, 2021). This has since been implemented in requirements for tenure and promotion, as well as in the template for performance review for both academic and non-academic staff. For the latter, a decision could be made to rename the three top leaves of the lotus flower model (research, education, professional performance) to reflect relevant other task domains, such as administration, information technology services or facility management. Opens in a new window.Fig. 4.1.TRIPLE Model for Recognition and Rewards, Utrecht University. For hiring and function profiles, the broadening of assessment can also translate into describing various types of academic functions, based on different profiles, with different sets of tasks and requiring different sets of skills and expertise. For instance, the University Medical Centre Utrecht (UMCU), an early mover in assessment reform, has introduced six academic career profiles: clinical researcher, academic educator, exploration researcher, implementation researcher, methodology and technology researcher and valorisation researcher (UMCU, 2022, 2023). Deepening Assessment As discussed above, broadening assessment from a narrow focus on research outcomes to a wider valuation of academic activities and contributions can help reduce pressure on single individuals to 'do everything' while being assessed primarily on research outcomes and can stimulate diversity in academic career paths by explicitly valuing all academic activities. In itself, though, this is no guarantee that for any given type of academic activity, quality and impact are appropriately assessed and, through that, encouraged. An important question then is: what are appropriate indicators for the broadening assessment of academic activities? The risk in this context is the use of proxy indicators, particularly quantitative indicators, for quality and impact. Well-known examples are the use of the JIF in research assessment as a proxy for both quality and impact or article-level citations as a proxy for quality (McKiernan et al., 2019). There are a number of risks associated with the use of such proxy indicators: the risk that the proxy does not measure what it is intended to measure (methodological risk); the risk that the proxy is used primarily because of its availability, not because of its relevance or methodological quality (streetlight effect); the risk of the proxy being used in isolation, without taking into account other indicators; and the risk that the proxy indicator, rather than the underlying quality, becomes the target-guiding practices of both academics and organisations (Goodhart's law). Taken together, the uncritical use of a limited set of proxy indicators can lead to perverse incentives (Hicks et al., 2015). This will be further elaborated on in Section 3 (Quantitative or Qualitative Assessment). In this section, we will address a number of approaches to move beyond such a limited approach. While we draw our examples primarily from research assessment as we are most involved and familiar with this domain, the same considerations and approaches are relevant for education and professional performance. One approach is to critically reflect on the indicators used: are they appropriate indicators for the purpose for which they are used? Are there other indicators that can complement or even replace the indicators used, e.g., to look at a broader set of outputs, and include a broader set of indicators for relevance and impact? For research assessment, this approach has been advocated by the San Francisco Declaration of Research Assessment (DORA) which recommends For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice. (DORA, 2013) Another aspect of 'deepening assessment' is to not look solely at outputs (be it of research, education or professional performance) but consider the activities and processes (including leadership and teamwork) leading to these outputs as subjects of assessment. One benefit of this is that assessment can be more formative: asking groups or individuals to reflect on their strategic goals and the activities undertaken to achieve these goals, as well as on the results thereof. In this way, assessment can bring about changes in process going forward, rather than be a reflection of 'success' or 'failure' after the fact. This is the approach taken by the Dutch Strategy Evaluation Protocol (VSNU, 2020) (formerly the 'Standard Evaluation Protocol', a telling rebranding it itself) in shaping the periodic formative evaluation of research groups. It allows for context-specific choices (accounting for disciplinary differences) as well as for choices to be made collectively at the level of the groups (research groups, departments, faculties) being evaluated. In addition, focusing on process shifts the focus from producing outputs (which can itself be a perverse incentive) to safeguarding good processes. Fig. 4.2 provides an overview of possible processes to include in research evaluation, with accompanying aspects that could be considered. Similar thought exercises could be envisioned for, e.g., education and professional performance. Opens in a new window.Fig. 4.2.Various Aspects of the Research Workflow That Could Be Considered in Assessment to Focus on Process Rather Than Outcomes. Finally, it is important to critically reflect on who sets the criteria for what is included in assessment and how assessment takes place. Especially when strategic goals are the starting point, the activities and outputs included in assessment, as well as any indicators used, should ideally be decided on in dialogue with who is assessed, rather than be decided for them. In addition, it should be carefully considered whether indicators used are appropriate for both the 'aggregation level' at which they are used and the goal of assessment. Assessment can take place at various levels: the individual, a research group or department, a university as a whole or even a whole country. It has already been discussed how a focus away from individuals and towards teams can allow for more diverse career paths and recognition of a wider spectrum of competencies (also reflecting a more inclusive, object-oriented approach to talent management). Assessment at the level of institutions or countries usually has a different role, more focused on comparing performance or understanding the effects of local differences. Depending on both the level at which assessment is taking place, and the goal of assessment, the use of certain indicators may not be appropriate. For example, institutional-level indicators do not reflect the qualities of individual people working or studying at that institution. Similarly, using indicators with the goal to increase understanding or even for promotional purposes carries lower risks for the entities being assessed than the use of metrics for incentivising or deciding on distribution of rewards (Gadd, 2019). Finally, assessment could also take into account various forms of hybridity, especially when institutions apply their renewed assessment goals and process to all staff, including support staff. Hybridity might involve people having mixed functions (e.g., part-time in an academic role and part-time in a support role), people switching between roles during their career (e.g., an academic moving into research policy for a few years and then back into research and teaching) and people being part of mixed project teams consisting of academic as well as support staff. All three have repercussions for choosing assessment criteria and for the design of the process, in particular the question who is involved in assessing. Qualitative or Quantitative Assessment One question that has been getting a lot of attention in the discussion around research assessment is the role of metrics versus peer review, sometimes put as a dichotomy between quantitative and qualitative evaluation. Peer review, defined by the European University Association (EUA) as the process of experts making a qualitative judgement of research quality (Saenen & Borrell-Damián, 2019), refers to the process where one or more individuals perform in-depth assessment, often followed by a consultation between the peer reviewers, or a comparison and synthesis of their assessments. Peer review can take place at various levels, both for assessing individual outputs (like research articles undergoing peer review before being published in a journal or grant proposals being assessed for funding), assessing individuals (for hiring tenure and promotion) and assessing research groups (like in the Dutch Strategy Evaluation Protocol (VSNU, 2020) which involves site visits). Peer review is sometimes considered the 'gold standard' – assuming assessment by a group of peers with knowledge of a specific discipline and context, can be expected to be more reliable than relying on metrics to make the 'right' decisions on, e.g., grant allocation, benchmarking research groups or hiring and promotion decisions. However, peer review has been shown to carry substantial variety in judgement between experts (peer reviewers) (Bertocchi et al., 2015; Cole et al., 1981; Traag & Waltman, 2019), raising questions on whether any decisions on, e.g., grant proposals objectively reflect the 'right' outcome, and even whether such an objectively right outcome exists in the first place (Lee et al., 2013). Other arguments against peer review that can be made are its sensitivity to subjective decisions (Teplitskiy et al., 2018), including the phenomenon that that search and hire commissions often gravitate towards candidates who are similar to them, as they are looking for a good 'fit' – leading to a lack of diversity (van den Brink & Benschop, 2014), as well as the argument that more qualitative methods often associated with peer review take a lot of time and are therefore sometimes considered unsustainable (Bendiscioli, 2018; Singh Chawla, 2019). In practice, assessment decisions made through peer review often already include the use of metrics or other indicators as part of the information gathered, and therefore, there is less of a dichotomy between qualitative and quantitative assessment, and more a question of what indicators are suitable for use in a given context. Also, both qualitative-based (review) and quantitative-based (metric) assessment reports can come with contextualisation and interpretation. As mentioned above, a few aspects to consider here are a) the validity of an indicator for the purpose it is used for; b) avoiding the 'streetlight' effect or choosing an indicator because it is available, rather than because it is the most appropriate; c) allowing a variety and diversity of indicators, rather than one or two default ones; and d) the risk that the indicator itself becomes a target (Goodhart's law). (Not) Fit for Purpose These aspects are all in play in cases where it is customary to use indicators which are not fit for purpose but which are used because they are readily available and commonly used by others and where there is resistance, distrust or just uncertainty towards using more diverse and less standardised indicators. Two examples of this are the use of the JIF and the h-index in assessing research quality. The JIF is a metric at the level of an academic journal, giving (roughly speaking) the average number of citations in a given year to papers published in the journal in the two preceding years. There are a number of issues with the use of JIF to assess research quality, both methodologically and conceptually (for a summary, see Larivière & Sugimoto, 2019, and Fig. 4.3). Arguably, the most important one is that, being an average at the journal level, the JIF does not reflect or predict the number of citations to any given (published or future) paper, as illustrated by the observed skewness of citation distribution in many journals (Larivière et al., 2016). A second, more general argument is that citations in themselves do not necessarily reflect quality and only reflect a particular type of impact (i.e., used as reference by other academics). Despite this, the JIF is used so ubiquitously in evaluations that publishing in high-impact journals has become a target in itself that shapes research practice (a prime example of Goodhart's law). Opens in a new window.Fig. 4.3.Why JIF Should Not Be Used to Assess Individual Researchers (Plomp et al., 2021). Somewhat similarly, the h-index (most often a metric at the level of individual researchers but which could also be applied at the level of journals or institutions) reflects the number of x publications (e.g., of a specific author) that each have received x or more citations. As such, it is a metric that favours late career over early career researchers (as the h-index can only rise over the course of a career) and again only reflects citations as at best a narrow metric of quality and impact. For an overview of the discussions around other problematic aspects of the use of the h-index, see Bornmann and Daniel (2009) and de Rijcke et al. (2021) and also Fig. 4.4. Opens in a new window.Fig. 4.4.Why h-Index Should Not Be Used to Assess Individual Researchers (Plomp et al., 2021). A compelling visual example of the various types of usage and impact of a researcher's output and activities that are disregarded when a narrow focus on JIF, h-index and citations in general is applied is provided in the infographic 'I am not my h-index (or my JIFs)' (Fig. 4.5), where against a background of a simple plot of number of publications and number of citations, a number of publications are highlighted with the specific impact they have had. Opens in a new window.Fig. 4.5.I Am Not My h-Index (or My JIFs) (Curry, 2018). Alternative Approaches At the surface level, examples of the use of broader indicators for research assessment include looking at indicators for societal impact (e.g., use of research in policy documents and public debate), looking at citation diversity, rather than citation counts, as a measure for the (academic) audience reached (Huang et al., 2022). At a deeper level, leaving the decision on which outputs and activities to report on, and which indicators to provide to demonstrate quality and impact, up to the individual researcher is an approach taken by the Dutch Research Council (NWO) in a number of its funding schemes (Dutch Research Council (now), n.d.; Gossink-Melenhorst, 2019). It is echoed in the Dutch Strategy Evaluation Protocol (VSNU, 2020) for the periodic evaluation of research groups at Dutch universities, which is aimed at aligning research evaluation with the aims and goals of those being assessed, rather than with a standardised concept of what counts as good performance. Finally, the explicit guidance provided by NWO to not use aggregate indicators (like JIF) to provide evidence of quality and impact of individual research outputs is a prime example of deepening research assessment – addressing inappropriate use of specific indicators. NWO also implemented the narrative, or evidence-based, CV – a combination of a narrative section to showcase the candidate's expertise and experience relevant to the project, and a 'key output' section to list a maximum of 10 outputs (not necessary publications) and indicators for their quality and impact. Other funders and institutions are also introducing the concept of narrative or evidence-based curriculum vitae as a way to enable qualitative and quantitative assessment in a contextualised way (Woolston, 2022), with the choices of what to present driven by those assessed rather than by those doing the assessment. It is important to stress that these formats still include objective indicators that can be assessed for their relevance and value by those performing assessments. Nonetheless, responses to narrative CVs have been mixed (Bordignon et al., 2023), with often-heard criticism that it favours those with the ability to 'sell themselves' on paper and fear that it will make assessment more subjective. Kaltenbrunner et al. (2023) have proposed a research agenda to get a better sense of the extent to which narrative CVs can be effective as part of a coordinated broader strategy to foster inclusive practices in research evaluation and of the practical conditions that must be met to achieve this potential. More in general, the right balance between relevance and contextualisation, on the one hand (with more options for those being assessed to select what to present to assessors), and comparability, on the other hand (with more standardised requirements for outputs and metrics to report and use), needs to be decided for any given assessment exercise. This

This dataset is part of the DIAMAS project deliverable 'IPSP Database'. The dataset contains a section of the DIAMAS survey where respondents authorised the DIAMAS project to include that information. After data cleaning and manual checking, this dataset contains data for 651 IPSPs. Data from Turkey will be added at a later stage. IPSP names are rendered in various languages. All IPSP URLs were tested with a Google Apps script, resulting in changes to 149 URLs, and the removal of three that did not resolve. IPSPs were further subdivided into ERA regions: Eastern Europe, Northern Europe, Southern Europe, Western Europe, Northern Africa, and Southwest Asia. IPSP responses from outside the ERA are out of scope for the report but included in the dataset in the region 'rest of the world'. The dataset provides the groundwork for the IPSP Registry, due to be delivered in June 2025 as part of DIAMAS Work Package 4: Building capacity through knowledge-sharing. All data are available for reuse under a CC0 license.

Ten years on from the launch of the Open Funder Registry (OFR, formerly FundRef), there is renewed interest in the potential of openly available funding metadata through Crossref. And with that: calls to improve the quality and completeness of that data. Currently, about 25% of Crossref records contain some kind of funding information. Over the years, this figure has grown steadily.

Research funders spend considerable efforts collecting information on the outcomes of the research they fund. To help funders track publication output associated with their funding, Crossref initiated FundRef in 2013, enabling publishers to register funding information using persistent identifiers. However, it is hard to assess the coverage of funder metadata because it is unknown how many articles are the result of funded research and should therefore include funder metadata. In this paper we looked at 5,004 publications reported by researchers to be the result of funding by a specific funding agency: the Dutch Research Council NWO. Only 67% of these articles contain funding information in Crossref, with a subset acknowledging NWO as funder name and/or Funder IDs linked to NWO (53% and 45%, respectively). Web of Science (WoS), Scopus, and Dimensions are all able to infer additional funding information from funding statements in the full text of the articles. Funding information in Lens largely corresponds to that in Crossref, with some additional funding information likely taken from PubMed. We observe interesting differences between publishers in the coverage and completeness of funding metadata in Crossref compared to proprietary databases, highlighting the potential to increase the quality of open metadata on funding.