Over the last several years, the computation landscape for conducting data analytics has completely changed. While in the past, a lot of the activities have been undertaken in isolation by companies, and research institutions, today's infrastructure constitutes a wealth of services offered by a variety of providers that offer opportunities for reuse, and interactions while leveraging service collaboration, and service cooperation. This document focuses on expanding analytics services to develop a framework for reusable hybrid multi-service data analytics. It includes (a) a short technology review that explicitly targets the intersection of hybrid multi-provider analytics services, (b) a small motivation based on use cases we looked at, (c) enhancing the concepts of services to showcase how hybrid, as well as multi-provider services can be integrated and reused via the proposed framework, (d) address analytics service composition, and (e) integrate container technologies to achieve state-of-the-art analytics service deployment

Today’s problems require a plethora of analytics tasks to be conducted to tackle state-of-the-art computational challenges posed in society impacting many areas including health care, automotive, banking, natural language processing, image detection, and many more data analytics-related tasks. Sharing existing analytics functions allows reuse and reduces overall effort. However, integrating deployment frameworks in the age of cloud computing are often out of reach for domain experts. Simple frameworks are needed that allow even non-experts to deploy and host services in the cloud. To avoid vendor lock-in, we require a generalized composable analytics service framework that allows users to integrate their services and those offered in clouds, not only by one, but by many cloud compute and service providers.We report on work that we conducted to provide a service integration framework for composing generalized analytics frame-works on multi-cloud providers that we call our Generalized AI Service (GAS) Generator. We demonstrate the framework’s usability by showcasing useful analytics workflows on various cloud providers, including AWS, Azure, and Google, and edge computing IoT devices. The examples are based on Scikit learn so they can be used in educational settings, replicated, and expanded upon. Benchmarks are used to compare the different services and showcase general replicability.

Big Data is the term used to describe the deluge of data in our networked, digitized, sensor-laden, information driven world. There is a broad agreement among commercial, academic, and government leaders about the remarkable potential of "Big Data" to spark innovation, fuel commerce, and drive progress. The availability of vast data resources carries the potential to answer questions previously out of reach. However, there is also broad agreement on the ability of Big Data to overwhelm traditional approaches. Big Data architectures come in many shapes and forms ranging from academic research settings to product-oriented workflows. With massive-scale dynamic data being generate from social media, Internet of Things, Smart Cities, and others, it is critical to analyze these data in real-time and provide proactive decision. With the advancement of computer architecture in multi-cores and GPUs, and fast communication between CPUs and GPUs, parallel processing utilizes these platforms could optimize resources at a reduced time. This presentation will provide the past, current, and future activities of the NIST Big Data Public Working Group (NBD-PWG) and how the NIST Reference Architecture may address the rate at which data volumes, speeds, and complexity are growing requires new forms of computing infrastructure to enable Big Data analytics interoperability such that analytics tools can be re-usable, deployable, and operational. The focus of NBD-PWG is to form a community of interest from industry, academia, and government, with the goal of developing consensus definitions, taxonomies, secure reference architectures, and standards roadmap which would create vendor-neutral, technology and infrastructure agnostic framework. The aim is to enable Big Data stakeholders to pick-and-choose best analytics tools for their processing under the most suitable computing platforms and clusters while allowing value-additions from Big Data service providers and flow of data between the stakeholders in a cohesive and secure manner.

Chapel is an emerging scalable, productive parallel programming language. In this work, we analyze Chapel's performance using The Parallel Research Kernels on two different manycore architectures including a state-of-the-art Intel Knights Landing processor. We discuss implementation techniques in Chapel and their relation to the OpenMP implementations of the PRK. We also suggest and prototype several optimizations in different layers of the software stack including the Chapel compiler. In our experiments we observed that base performance of Chapel ranges from 41%-184% that of OpenMP. The optimization techniques we discussed shows performance improvements ranging from 1.4x to 2x in Chapel.

This article examines foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new data science research program (DSRP) and associated data science evaluation (DSE) series, introduced by the National Institute of Standards and Technology (NIST) in the fall of 2015. The DSRP is designed to facilitate and accelerate research progress in the field of data science and consists of four components: evaluation and metrology, standards, compute infrastructure, and community outreach. A key part of the evaluation and measurement component is the DSE. The DSE series aims to address logistical and evaluation design challenges while providing rigorous measurement methods and an emphasis on generalizability rather than domain- and application-specific approaches. Toward that end, each year the DSE will consist of multiple research tracks and will encourage the application of tasks that span these tracks. The evaluations are intended to facilitate research efforts and collaboration, leverage shared infrastructure, and effectively address crosscutting challenges faced by diverse data science communities. Multiple research tracks will be championed by members of the data science community with the goal of enabling rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will permit us to measure several different data science technologies in a wide range of fields and will address computing infrastructure, standards for an interoperability framework, and domain-specific examples. This article also summarizes lessons learned from the data science evaluation series pre-pilot that was held in fall of 2015.

Big Data is a term used to describe the large amount of data in the networked, digitized, sensor-laden, information-driven world. While opportunities exist with Big Data, the data can overwhelm traditional technical approaches, and the growth of data is outpacing scientific and technological advances in data analytics. To advance progress in Big Data, the NIST Big Data Public Working Group (NBD-PWG) is working to develop consensus on important fundamental concepts related to Big Data. The results are reported in the NIST Big Data Interoperability Framework (NBDIF) series of volumes. This volume, Volume 6, summarizes the work performed by the NBD-PWG to characterize Big Data from an architecture perspective, presents the NIST Big Data Reference Architecture (NBDRA) conceptual model, discusses the roles and fabrics of the NBDRA, presents an activities view of the NBDRA to describe the activities performed by the roles, and presents a functional component view of the NBDRA containing the classes of functional components that carry out the activities.

The Information Access Division (IAD) of the National Institute of Standards and Technology (NIST) launched a new Data Science Initiative in the fall of 2015. This initiative focuses on evaluation-driven research and will establish a new Data Science Evaluation series to facilitate research collaboration, to leverage shared technology and infrastructure, and to further build and strengthen the data science community. The evaluation series will consist of a pre-pilot to be launched in the fall of 2015, a pilot evaluation to be launched in 2016, and a full-scale multiple-track evaluation in 2017. In addition to these evaluations, this new initiative aims to address several infrastructure challenges and to provide standards to encourage easier group collaboration.

We examine foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new Data Science Initiative and evaluation series, introduced by the National Institute of Standards and Technology (NIST) in the fall of 2015. The evaluations will facilitate research efforts, collaboration, leverage shared infrastructure, and effectively address cross-cutting challenges faced by diverse data science communities. The evaluations will have multiple research tracks championed by members of the data science community, and will enable rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will measure several different data science technologies in a wide range of fields, starting with a pre-pilot. In addition to developing data science evaluation methods and metrics, it will address computing infrastructure, standards for an interoperability framework, and domain-specific examples.

We examine foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new Data Science Research Program and evaluation series, introduced by the Information Access Division (IAD) of the National Institute of Standards and Technology (NIST) in the fall of 2015. The evaluations will facilitate research efforts, collaboration, leverage shared infrastructure, and effectively address cross-cutting challenges faced by diverse data science communities. The evaluations will have multiple research tracks championed by members of the data science community, and will enable rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will measure several different data science technologies in a wide range of fields, starting with a pre-pilot. In addition to developing data science evaluation methods and metrics, it will address computing infrastructure, standards for an interoperability framework, and domain-specific examples.

Big Data has changed both technologies and practices for building data analytics systems. A number of working groups have been discussing the recent changes along a number of dimensions. The NIST Big Data Public Working Group organized a workshop to promote communication among working groups, technologists, and practitioners to come to an understanding of the current state of the Big Data discipline, collaboration best practices, future directions for this emerging specialization, and to identify security and privacy concerns.

This paper introduces the motivation and describes the model of a novel metadata standard for the exchange of preservation metadata of multimedia content. The model is being standardised as the MPEG Multimedia Preservation Application Format (MPAF), addressing the specific issues related to the preservation description information of audiovisual contents. Several standards for expressing metadata in digital preservation are available and have been taken into consideration in the paper and in the work done in MPEG. However, none of them is able to cover all the needed aspects related to the preservation of audiovisual content. Audiovisual files are in most cases containers and are usually made up of several tracks carrying audio data, video data and specific time-based metadata. In order to be able to perform the opportune preservation actions (among others planning and format migration), several kinds of information must be kept alongside the audiovisual contents. Within this context a standardised representation for these structures and metadata is needed. Information such as quality description or fixity at frame level is required for ensuring long term access to visual content. Without using a standardised interface it is hard to guarantee a faithful rendering of encoded information while exchanging contents between different repositories, either internally or with external institutions. This paper describes the work done so far within MPEG for defining a standard metadata model which covers the identified missing parts and gaps regarding the acquisition of digital preservation description information.