We have created tools that automate one of the most burdensome aspects of documenting the provenance of research data: describing data transformations performed by statistical software. Researchers in many fields use statistical software (SPSS, Stata, SAS, R, Python) for data transformation and data management as well as analysis. The C2Metadata ("Continuous Capture of Metadata for Statistical Data") Project creates a metadata workflow paralleling the data management process by deriving provenance information from scripts used to manage and transform data. C2Metadata differs from most previous data provenance initiatives by documenting transformations at the variable level rather than describing a sequence of opaque programs. Command scripts for statistical software are translated into an independent Structured Data Transformation Language (SDTL), which serves as an intermediate language for describing data transformations. SDTL can be used to add variable-level provenance to data catalogues and codebooks and to create "variable lineages" for auditing software operations. Better data documentation makes research more transparent and expands the discovery and re-use of research data.

Structured Data Transformation Language (SDTL) provides structured, machine actionable representations of data transformation commands found in statistical analysis software. The Continuous Capture of Metadata for Statistical Data Project (C2Metadata) created SDTL as part of an automated system that captures provenance metadata from data transformation scripts and adds variable derivations to standard metadata files. SDTL also has potential for auditing scripts and for translating scripts between languages. SDTL is expressed in a set of JSON schemas, which are machine actionable and easily serialized to other formats. Statistical software languages have a number of special features that have been carried into SDTL. We explain how SDTL handles differences among statistical languages and complex operations, such as merging files and reshaping data tables from “wide” to “long”.

Supported by the Data Infrastructure Building Blocks (DIBBs) program of the National Science Foundation through grant NSF ACI-1640575.

Supported by the Data Infrastructure Building Blocks (DIBBs) program of the National Science Foundation through grant NSF ACI-1640575.

Creating Rich, Structured Metadata: Lessons Learned in the Metadata Portal Project

This Master thesis examines issues of interoperability and integration between the Classic Information Science (CIS) and Quantum Information Science (QIS). It provides a short introduction to the Extensible Markup Language (XML) and proceeds to describe the development steps that have lead to a prototype XML specification for quantum computing (QIS-XML). QIS-XML is a proposed framework, based on the widely used standard (XML) to describe, visualize, exchange and process quantum gates and quantum circuits. It also provides a potential approach to a generic programming language for quantum computers through the concept of XML driven compilers. Examples are provided for the description of commonly used quantum gates and circuits, accompanied with tools to visualize them in standard web browsers. An algorithmic example is also presented, performing a simple addition operation with quantum circuits and running the program on a quantum computer simulator. Overall, this initial effort demonstrates how XML technologies could be at the core of the architecture for describing and programming quantum computers. By leveraging a widely accepted standard, QIS-XML also builds a bridge between classic and quantum IT, which could foster the acceptance of QIS by the ICT community and facilitate the understanding of quantum technology by IT experts. This would support the consolidation of Classic Information Science and Quantum Information Science into a Complete Information Science, a challenge that could be referred to as the "Information Science Grand Unification Challenge".

An unprecedented amount of data are produced by agencies and researchers in support of SBE research. However, the existence of data alone is not sufficient to maximize their utility and the need for more accessible, open, and usable data is growing. Meeting such demand is a major challenge for data managing institutions that are, not only facing a number of archetypal issues - such as quality, timeliness, adequate documentation, privacy, security, comparability, and harmonization - but also significant resource constraints. Furthermore, the globalization and virtualization of data and systems calls for a unification of practices to facilitate the process by which information may be delivered over the Internet or exchanged between organizations. New information technologies, tools, best practices, and metadata standards have emerged to meet these needs in a way that is both efficient and responsible, the adoption and diffusion of which can provide producers and researchers with a cohesive approach to data management and knowledge sharing. This in turn will open greater access to data, maximizing their usefulness for policymakers, students, researchers, and journalists. This paper examines how advances in technology and open standards may be leveraged upon to enhance manageability, discovery, accessibility, and usability of SBE data.