In an era where astronomical data is expanding at an unprecedented rate, the importance of data sharing and accessibility among astronomy archives cannot be overstated. Since the 1990s, an international partnership between the Space Telescope Science Institute (STScI), the European Space Astronomy Centre (ESAC), and the Canadian Astronomy Data Centre (CADC) has been focused on this endeavor, facilitating the exchange of data from the Hubble and James Webb Space Telescopes. We will present how this collaboration has evolved over time, highlighting key milestones and innovations in decision-making, communication, and technology. Additionally, we will discuss some of the challenges we have encountered and the strategies we employed to overcome them, offering insights that could benefit future archive collaborations.

Initiated in 2008, the Canadian Advanced Network for Astronomical Research (CANFAR) has evolved into an advanced science platform, a cloud-native framework for remote analysis of astronomical data. This innovative platform is designed for a diverse range of users, from large-scale groups like CHIME-FRB, to individual researchers in remote institutions. It offers intuitive interfaces such as notebooks, desktops, visualizers and IDEs, all accessible through a browser. These are supported by multi-tiered storage and Kubernetes orchestration. A comprehensive REST API facilitates seamless integration and automated batch processing. CANFAR stands out by combining conventional desktop analysis with cutting-edge, browser-based interfaces and GPU-accelerated machine learning capabilities. This unique blend has made it a hub for a varied user community, establishing it as a unified platform for comprehensive astronomical data analysis.

The International Virtual Observatory Alliance (IVOA) held its bi-annual Interoperability Meeting over two and half days prior to the ADASS 2018 conference. We provide a brief report on the status of the IVOA and the activities of the Interoperability Meeting held in College Park.

A joint project between the Canadian Astronomy Data Center of the National Research Council Canada, and the italian Istituto Nazionale di Astrofisica-Osservatorio Astronomico di Trieste (INAF-OATs), partially funded by the EGI-Engage H2020 European Project, is devoted to deploy an integrated infrastructure, based on the International Virtual Observatory Alliance (IVOA) standards, to access and exploit astronomical data. Currently CADC-CANFAR provides scientists with an access, storage and computation facility, based on software libraries implementing a set of standards developed by the International Virtual Observatory Alliance (IVOA). The deployment of a twin infrastructure, basically built on the same open source software libraries, has been started at INAF-OATs. This new infrastructure now provides users with an Access Control Service and a Storage Service. The final goal of the ongoing project is to build an integrated infrastructure geographycally distributed providing complete interoperability, both in users access control and data sharing. This paper describes the target infrastructure, the main user requirements covered, the technical choices and the implemented solutions.

The Canadian Astronomy Data Centre (CADC) and the European Space Astronomy Centre (ESAC) maintain mirrors of and provide user access to the HST data collection. A new mirroring approach was needed to improve consistency and support future missions like JWST. The Common Archive Observation Model (CAOM) is used as the core model for all data holdings at the CADC and the Mikulski Archive for Space Telescopes (MAST) and was extended to support a metadata and data synchronization system. The metadata synchronization process relies on a simple RESTful web service operated by the metadata source (MAST) and a metadata harvesting tool run by the mirror centers (CADC and ESAC). The data synchronization process uses CAOM metadata to discover and retrieve files from the source (MAST) to the mirror sites. Through the use a backend plugins, the CADC and ESAC have extended the file synchronization tool to interface with their respective site-specific storage systems. Using the common metadata model, services and tools as a base, partners can augment their own system with additional information specifically intended to provide added value features. ESAC provides information about publications in their instance of the Archive and both CADC and ESAC provide additional IVOA services for users.

Many astronomy projects today are executed by distributed science teams with access to different computation and storage resources. As we move into the era of petabyte and exabyte datasets, it is recognized that moving the code to the data becomes necessary as the alternative becomes infeasible. The question becomes how can resource infrastructures support these large projects such that a team has integrated access to the different distributed resources available to a project. Examples of resources that could be integrated are files and directories, storage allocations, processing allocations, containers and virtual machine images, databases and tables, etc. A first step in this direction is the interoperability of authorization services. The International Virtual Observatory Alliance (IVOA) has developed many standards to support access and interoperability of infrastructure such as Single-Sign On (SSO), Credential Delegation Protocol (CDP) and VOSpace. Both Canadian Advanced Network for Astronomical Research (CANFAR) operated by the Canadian Astronomy Data Centre) and INAF-Osservatorio Astronomico di Trieste (INAF-OAT) use these standards for provision of user storage to support projects. In the VOSpace implementation, users assign read-only and read/write permissions to groups that are defined in their respective home institution Group Management Services. In 2015, the EGI-Engage project in Europe partially funded an exploration of interoperability of authorization services in a joint project between the CANFAR and INAF-OAT. This has also led to the inclusion of this work in the Advanced European Network of E-infrastructures for Astronomy with the SKA (Aeneas) proposal. The joint CANFAR/ INAF-OAT project has added support to interoperate its VOSpace services by adding the capability of granting authorization to access a resource to groups defined in an external Group Management Service and to allow for the dynamic creation of internal user IDs that are associated with an external identify provider.

The Common Archive Observation Model (CAOM) is used as the core data model for all data holdings at the Canadian Astronomy Data Centre (CADC): 12 million data products spanning 20 collections and including images, spectra, cubes, time series, and catalogues. The CADC supports both in-house and external data engineering efforts by providing client tools and operating a CAOM observation repository service where data engineers have permission to create, update, and delete observation metadata. CAOM supports anonymous access for public data and authenticated access to proprietary metadata and data. The content is loosely coupled to data storage and delivery system. In addition to the CADC AdvancedSearch (web portal), CAOM also supports International Virtual Observatory Alliance (IVOA) services and data models to provide an end-to-end solution for data discovery and data access. CAOM is an open source model and open source codebase that is evolving along with the demands of both data variety and the needs of science users, science teams, and data centre operations.

This document defines a standard modelling language, or meta-model, for expressing data models in the IVOA. Adopting such a uniform language for all models allows these to be used in a homogeneous manner and allows a consistent definition of reuse of one model by another. The particular language defined here includes a consistent identification mechanism for model which

Handling, processing and archiving the huge amount of data produced by the new generation of experiments and instruments in Astronomy and Astrophysics are among the more exciting challenges to address in designing the future data management infrastructures and computing services. We investigated the feasibility of a data management and computation infrastructure, available world-wide, with the aim of merging the FAIR data management provided by IVOA standards with the efficiency and reliability of a cloud approach. Our work involved the Canadian Advanced Network for Astronomy Research (CANFAR) infrastructure and the European EGI federated cloud (EFC). We designed and deployed a pilot data management and computation infrastructure that provides IVOA-compliant VOSpace storage resources and wide access to interoperable federated clouds. In this paper, we detail the main user requirements covered, the technical choices and the implemented solutions and we describe the resulting Hybrid cloud Worldwide infrastructure, its benefits and limitations. Crown Copyright (C) 2018 Published by Elsevier B.V. All rights reserved.

The increase of astronomical data produced by a new generation of observational tools poses the need to distribute data and to bring computation close to the data. Trying to answer this need, we set up a federated data and computing infrastructure involving an international cloud facility, EGI federated, and a set of services implementing IVOA standards and recommendations for authentication, data sharing and resource access. In this paper we describe technical problems faced, specifically we show the designing, technological and architectural solutions adopted. We depict our technological overall solution to bring data close to computation resources. Besides the adopted solutions, we propose some points for an open discussion on authentication and authorization mechanisms.

This document describes the Server-side Operations for Data Access (SODA) web service capability. SODA is a low-level data access capability or server side data processing that can act upon the data files, performing various kinds of operations: filtering/subsection, transformations, pixel operations, and applying functions to the data.

The IVOA Data Access Layer (DAL) working group was created in 2002 to define protocols to homogenize data discovery, data description, data retrieval, and data access processes. We describe its history and status today, and look at current trends for future development of the DAL protocols.

The Simple Image Access protocol (SIA) provides capabilities for the discovery, description, access, and retrieval of multi-dimensional image datasets, including 2-D images as well as datacubes of three or more dimensions. SIA data discovery is based on the ObsCore Data Model (ObsCoreDM), which primarily describes data products by the physical axes (spatial, spectral, time, and polarization). Image datasets with dimension greater than 2 are often referred to as datacubes, cube or image cube datasets and may be considered examples of hypercube or n-cube data. In this document the term "image" refers to general multi-dimensional datasets and is synonymous with these other terms unless the image dimensionality is otherwise specified. SIA provides capabilities for image discovery and access. Data discovery and metadata access (using ObsCoreDM) are defined here. The capabilities for drilling down to data files (and related resources) and services for remote access are defined elsewhere, but SIA also allows for direct access to retrieval.

This document describes the linking of data discovery metadata to access to the data itself, further detailed metadata, related resources, and to services that perform operations on the data. The web service capability supports a drill-down into the details of a specific dataset and provides a set of links to the dataset file(s) and related resources. This specification also includes a VOTable-specific method of providing descriptions of one or more services and their input(s), usually using parameter values from elsewhere in the VOTable document. Providers are able to describe services that are relevant to the records (usually datasets with identifiers) by including service descriptors in a result document.

We present, as a case study, a description of the partnership between an observatory (JCMT) and a data centre (CADC) that led to the development of the JCMT Science Archive (JSA). The JSA is a successful example of a service designed to use Virtual Observatory (VO) technologies from the start. We describe the motivation, process and lessons learned from this approach.

This document describes an XML encoding standard for metadata about services implementing the table access protocol TAP [TAP], referred to as TAPRegExt. Instance documents are part of the service's registry record or can be obtained from the service itself. They deliver information to both humans and software on the languages, output formats, and upload methods supported by the service, as well as data models implemented by the exposed tables, optional language features, and certain limits enforced by the service.

This document describes the Data Access Layer Interface (DALI). DALI defines the base web service interface common to all Data Access Layer (DAL) services. This standard defines the behaviour of common resources, the meaning and use of common parameters, success and error responses, and DAL service registration. The goal of this specification is to define the common elements that are shared across DAL services in order to foster consistency across concrete DAL service specifications and to enable standard re-usable client and service implementations and libraries to be written and widely adopted.

The Group Membership Service (GMS), implemented at the Canadian Astronomy Data Centre (CADC), is a prototype of what could eventually be an IVOA standard for a distributed and interoperable group membership protocol.Group membership is the core authorization concept that enables teamwork and collaboration amongst astronomers accessing distributed resources and services. The service integrates and complements other access control related IVOA standards such as single-sign-on (SSO) using X.509 proxy certificates and the Credential Delegation Protocol (CDP). The GMS has been used at CADC for several years now, initially as a subsystem and then as a stand-alone Web service. It is part of the authorization mechanism for controlling the access to restricted Web resources as well as the VOSpace service hosted by the CADC. We present the role that GMS plays within the access control system at the CADC, including the functionality of the service and how the different CADC services make use of it to assert user authorization to resources. We also describe the main advantages and challenges of using the service as well as future work to increase its robustness and functionality.