On 18 November 2025, a small error during internal data migration of between storage systems of the JASMIN data analysis platform in the UK led to a substantial part of the CEDA Archive being made temporarily unavailable online (but not lost!). The unfortunate incident caused serious disruption to a large community of users (and additional workload and stress for the team), it provided important learning points for the team in terms of: enhancing data security, importance of mutual support among professional colleagues, the value of clear and transparent communications with your users a unique opportunity to showcase the capabilities of a cutting-edge digital research infrastructure in the recovery and return to service with this “unscheduled disaster recovery exercise”. We report on the circumstances leading to the incident, the lessons learned, and the technical capabilities employed in the recovery. One example shows, nearly 800 Terabytes of data transferred from a partner institution in the USA in just over 27 hours, at a rate of over 8 Gigabytes per second using Globus. The ability to orchestrate such a transfer is the result of many years of international collaboration to support large-scale environmental science, and highlights the benefits of a federated, replicated data infrastructure built on well-engineered technologies.

The Natural Environment Research Council Environmental Data Service (NERC EDS) provides integrated data services across the breadth of NERC’s data holdings and coordinates closer collaboration and development between NERC’s five environmental data centres. Data is central to modern understanding of our environment. Environmental science is underpinned by access to high quality sources of data and data services. As the principal funder of environmental science in the UK, NERC has supported comprehensive data services and policies since its creation over 50 years ago. Today NERC has five Environmental Data Centres embedded within its Research Centres:• The British Oceanographic Data Centre (BODC) provides data and services across marine science and is embedded within the National Oceanography Centre.• The Centre for Environmental Data Analysis (CEDA) provides both atmospheric and Earth Observation data and is embedded within the National Centre for Atmospheric Science and the National Centre for Earth Observation.• The Environmental Information Data Centre (EIDC) supports the data requirements of the terrestrial and freshwater sciences and is embedded within the UK Centre for Ecology and Hydrology.• The UK Polar Data Centre (PDC) is responsible for all of the UK’s polar data holdings and associated services and is embedded within the British Antarctic Survey.• The National Geoscience Data Centre (NGDC) provides geoscience and subsurface data and services and is embedded within the British Geological Survey.Each of the five environmental data centres specialises in data within a particular sub-discipline of environmental science and serves not only NERC’s science community but also a much broader community of users and stakeholders, which span research, industry, education, government and voluntary organisations. At the same time, science and its applications are becoming increasingly multi-disciplinary. Often users of NERC data will need to access the services provided by multiple data centres, they will also need to be interoperable in a European and global context.In order to serve an ever-growing community of data users and stakeholders, in 2018 NERC created its Environmental Data Service to coordinate across the data centres. During its first five years the EDS has provided growing coordination between NERC’s data centres, both for user services and the development of new, discipline-independent services.NERC has recently recommissioned its data services for the period 2023-2028. As a consequence, the EDS is embarking on an ambitious plan to deliver increasingly integrated services across the full breadth of NERC science and meeting the environmental data needs of stakeholders and users across the UK and beyond. This will require further development of common back-end services and front-end development of FAIR practices including standardised vocabularies and ontologies to support both disciplinary science and wider stakeholder engagement but increasingly an approach to transdiciplinarity to facilitate both next generation science and approaches to foster wider data engagement in responding to the grand societal challenges.

Environmental science is concerned with assessing the impacts of changing environmental conditions upon the state of the natural world. Environmental Digital Twins (EDT) are a new technology that enable environmental change scenarios for real systems to be modelled and their impacts visualised. They will be particularly effective with delivering understanding of these impacts on the natural environment to non-specialist stakeholders. The UK Natural Environment Research Council (NERC) recently published its first digital strategy, which sets out a vision for digitally enabled environmental science for the next decade. This strategy places data and digital technologies at the heart of UK environmental science. EDT have been made possible by the emergence of increasingly large, diverse, static data sources, networks of dynamic environmental data from sensor networks and time-variant process modelling. Once combined with visualisation capabilities these provide the basis of the digital twin technologies to enable the environmental scientists community to make a step-change in understanding of the environment. Components may be developed separately by a network but can be combined to improve understanding provided development follows agreed standards to facilitate data exchange and integration. Replicating the behaviours of environmental systems is inevitably a multi-disciplinary activity. To enable this, an information management framework for Environmental digital twins (IMFe) is needed that establishes the components for effective information management within and across the EDT ecosystem. This must enable secure, resilient interoperability of data, and is a reference point to facilitate data use in line with security, legal, commercial, privacy and other relevant concerns. We present recommendations for developing an IMFe including the application of concepts such as an asset commons and balanced approach to standards to facilitate minimum interoperability requirements between twins while iteratively implementing an IMFe. Achieving this requires components to be developed that follow agreed standards to ensure that information can be trusted by the user, and that they are semantically interoperable so data can be shared. A digital Asset Register will be defined to provide access to and enable linking of such components. This previously conceptual project has now been enhanced into the Pilot IMFe project aiming to define the architectures, technologies, standards and hardware infrastructure to develop a fully functioned environmental digital twin. During the project lifespan this will be tested with by construction of a pilot EDT for the Haig Fras Marine Conservation Zone (MCZ) that both enables testing of the proposed IMFe concepts and will provide a clear demonstration of the power of EDT to monitor and scenario test a complex environmental system for the benefit of stakeholders.

JASMIN is a highly successful data analysis system, which is used by thousands of academics and their industrial partners to analyse many petabytes of environmental data. The rapidly increasing volume of data stored on JASMIN, and the steadily increasing number of users, is making it necessary to investigate and implement new methods of providing computing resources to the users, storing the data that they produce from their analyses and storing and maintaining a very large archive of environmental data. To achieve this, two main areas of research are described. Firstly, providing users with virtualised services to best utilise the computing resources available. Secondly, using object storage to provide a large, yet affordable, data store and providing the users with tools and interfaces to common environmental data formats, so as to not unduly affect their current work flows.

In this paper we decribe a business process modelling approach to the integration of exisiting archival activities. We provide a high level overview of existing practice and discuss how procedures can be extended and supported through the description of preservation state. The aim of which is to faciliate the dynamic controlled management of scientific data through its lifecycle. The main types of archival processes considered are: • Management processes that govern the operation of an archive. These management processes include archival governance (preservation state management, selection of archival candidates and strategic management) . • Operational processes that constitute the core activities of the archive which maintain the value of research assets. These operational processes are the acquisition, ingestion, deletion, generation of metadata and preservation actvities, • Supporting processes, which include planning, risk analysis and monitoring of the community/preservation environment.

The British Atmospheric Data Centre (BADC) has existed in its present form for 20 years, having been formally created in 1994. It evolved from the GDF (Geophysical Data Facility), a SERC (Science and Engineering Research Council) facility, as a result of research council reform where NERC (Natural Environment Research Council) extended its remit to cover atmospheric data below 10km altitude. With that change the BADC took on data from many other atmospheric sources and started interacting with NERC research programmes. The BADC has now hit early adulthood. Prompted by this milestone, we examine in this paper whether the data centre is creaking at the seams or is looking forward to the prime of its life, gliding effortlessly into the future. Which parts of it are bullet proof and which parts are held together with double-sided sticky tape? Can we expect to see it in its present form in another twenty years’ time? To answer these questions, we examine the interfaces, technology, processes and organisation used in the provision of data centre services by looking at three snapshots in time, 1994, 2004 and 2014, using metrics and reports from the time to compare and contrasts the services using BADC. The repository landscape has changed massively over this period and has moved the focus for technology and development as the broader community followed emerging trends, standards and ways of working. The incorporation of these new ideas has been both a blessing and a curse, providing the data centre staff with plenty of challenges and opportunities. We also discuss key data centre functions including: data discovery, data access, ingestion, data management planning, preservation plans, agreements/licences and data policy, storage and server technology, organisation and funding, and user management. We conclude that the data centre will probably still exist in some form in 2024 and that it will most likely still be reliant on a file system. However, the technology delivering this service will change and the host organisation and funding routes may vary.

Course material used during the NCAS Introduction to Scientific Computing Course, University of York, March 17-20 2014

JASMIN is a super-data-cluster designed to provide a high-performance high-volume data analysis environment for the UK environmental science community. Thus far JASMIN has been used primarily by the atmospheric science and earth observation communities, both to support their direct scientific workflow, and the curation of data products in the STFC Centre for Environmental Data Archival (CEDA). Initial JASMIN configuration and first experiences are reported here. Useful improvements in scientific workflow are presented. It is clear from the explosive growth in stored data and use that there was a pent up demand for a suitable big-data analysis environment. This demand is not yet satisfied, in part because JASMIN does not yet have enough compute, the storage is fully allocated, and not all software needs are met. Plans to address these constraints are introduced.