We introduce PathQuery, a graph query language developed to scale with Google's query and data volumes as well as its internal developer community. PathQuery supports flexible and declarative semantics. We have found that this enables query developers to think in a naturally "graphy" design space and to avoid the additional cognitive effort of coordinating numerous joins and subqueries often required to express an equivalent query in a relational space. Despite its traversal-oriented syntactic style, PathQuery has a foundation on a custom variant of relational algebra -- the exposition of which we presently defer -- allowing for the application of both common and novel optimizations. We believe that PathQuery has withstood a "test of time" at Google, under both large scale and low latency requirements. We thus share herein a language design that admits a rigorous declarative semantics, has scaled well in practice, and provides a natural syntax for graph traversals while also admitting complex graph patterns.

The a mount of data produced in support of scientific research continues to grow rapidly. Despite the accumulation and demand for scientific data, relatively little data are actually made available for the broader scientific community. We surmise that one root of this problem is the perceived difficulty of electronically publishing scientific data and associated metadata in a way that makes it discoverable. We propose exploiting Semantic Web technologies and best practices to make metadata both discoverable and easy to publish. We share experiences in curating metadata to illustrate the cumbersome nature of data reuse in the current research environment. We also make recommendations with a real-world example of how data publishers can provide their metadata by adding limited additional markup to HTML pages on the Web. With little additional effort from data publishers, the difficulty of data discovery, access, and sharing can be greatly reduced and the impact of research data greatly enhanced.

This paper presents GEMS (Graph database Engine for Multithreaded Systems), a software infrastructure that enables large-scale, graph databases on commodity clusters. Unlike current approaches, GEMS implements and explores graph databases by primarily employing graph-based methods. This is reflected in all the layers of the software stack. On one hand, this allows exploiting the space efficiency of graph data structures and the inherent parallelism of some graph algorithms. These features adapt well to the increasing system memory and core counts of modern commodity clusters. On the other hand, however, graph-based methods introduce irregular, finegrained data accesses with poor spatial and temporal locality, causing performance issues with these systems that are, instead, optimized for regular computation and batched data transfers. Our framework comprises: a SPARQL to data parallel C++ compiler; a library of distributed data structures; and a custom, multithreaded, runtime system. We introduce our stack, motivate its advantages with respect to other solutions, and show how we solved the challenges posed by irregular behaviors. We evaluate our software stack on the Berlin SPARQL benchmark with datasets up to 10 billion graph edges, demonstrating scaling in dataset size and in performance as nodes are added to a cluster.

A software stack relies primarily on graph-based methods to implement scalable resource description framework databases on top of commodity clusters, providing an inexpensive way to extract meaning from volumes of heterogeneous data.

In this work we propose a novel approach for RDF (Resource Description Framework) dictionary encoding that employs a parallel RDF parser and a distributed dictionary data structure, exploiting RDF-specific optimizations. In contrast with previous solutions, this approach exploits the Partitioned Global Address Space (PGAS) programming model combined with active messages. We evaluate the performance of our dictionary encoder in our RDF database, GEMS (Graph Engine for Multithreaded Systems), and provide an empirical comparison against previous approaches. Our comparison shows that our dictionary encoder scales significantly better and achieves higher performance than the current state of the art, providing a key element for the realization of a more efficient RDF database.

This chapter discusses the approaches integrated in GEMS (Graph database Engine for Multithreaded Systems) for managing and querying datasets of RDF (Resource Description Framework) triples. GEMS is a software stack that implements graph databases on top of commodity, high-performance clusters. GEMS is composed of a SPARQL-to-C++ compiler, a library of data structures and parallel graph methods, and a multithreaded runtime library. Differently from other RDF databases, which resort to more conventional relational databases approaches, and largely employ table-based methods for query processing, GEMS mostly employs graph methods. Query processing in GEMS is performed through the conversion of SPARQL queries into graph homomorphism routines, which then are directly executed on the graph database (resulting from the RDF triples ingestion) through its runtime library. The runtime library, which implements a Partitioned Global Address Space (PGAS), lightweight software multithreading, and network message aggregation, mitigates some of the typical issues of graph processing on modern commodity clusters, enabling scaling in performance and size while new cluster nodes are added. In fact, although very powerful, these systems are built for regular computation and easily partitionable workloads, while graph processing typically has an irregular behavior. The chapter explains how SPARQL queries can be naturally modeled as graph pattern-matching algorithms and details how GEMS performs the conversion to C++ routines. It briefly discusses the other components of GEMS and then shows the results of the full stack on the Berlin SPARQL Benchmark (BSBM) and the SPARQL Performance Benchmark (SP2B). We discuss effects of the automatic conversion and present a comparison with a full custom appliance for data analytics (YarcData Urika).

GEMS IS A FULL SOFTWARE SYSTEM THAT IMPLEMENTS A LARGE-SCALE, SEMANTIC GRAPH DATABASE ON COMMODITY CLUSTERS. ITS FRAMEWORK COMPRISES A SPARQL-TO-C++ COMPILER, A LIBRARY OF DISTRIBUTED DATA STRUCTURES, AND A CUSTOM MULTITHREADED RUNTIME LIBRARY. THE AUTHORS EVALUATED THEIR SOFTWARE STACK ON THE BERLIN SPARQL BENCHMARK WITH DATASETS OF UP TO 10 BILLION GRAPH EDGES, DEMONSTRATING SCALING IN DATASET SIZE AND PERFORMANCE AS THEY ADDED CLUSTER NODES.

We are developing a full software system for accelerating semantic graph databases on commodity cluster that scales to hundreds of nodes while maintaining constant query throughput. Our framework comprises a SPARQL to C++ compiler, a library of parallel graph methods and a custom multithreaded runtime layer, which provides a Partitioned Global Address Space (PGAS) programming model with fork/join parallelism and automatic load balancing over a commodity clusters. We present preliminary results for the compiler and for the runtime.

This thesis considers the problem of scaling rule-based inference to large quantities of RDF data found on the Semantic Web. The general approach is one of data parallelism, that is, dividing data among processors such that the collective results of each processor's individual inference is the same as though inference was performed sequentially. In this way, theoretically speaking, more processors can be added to accommodate more data. The problem is first considered from the perspective of the operational semantics of inference with production rules. The question is asked, under what conditions is embarrassingly parallel inference guaranteed to be correct? Sufficient conditions are determined and proven at both a fine-grained level close to the basic operational semantics and a more coarse-grained level that applies directly to rules. The conditions are placed on the relationship between rules and distribution schemes, that is, the way in which data is assigned to processors. Then, a special class of distribution schemes is considered called replication schemes. Replication schemes require that individual data either be replicated to all processors or placed arbitrarily on some processor(s). The aforementioned conditions are then reformulated to consider replication schemes which reveals that testing the conditions for replication schemes is reducible to satisfiability (SAT), and not only SAT but 2SAT. An augmented version of this reduction which is a reduction to 3SAT also accounts for the possibility to eliminate some rules in order to improve parallelization. These reductions along with a proposed methodology for restricting rules are used to derive restricted versions of the RDFS and OWL2RL rules that are amenable to parallel inference. Finally, an evaluation is performed that tests these theoretical findings for restricted versions of RDFS and OWL2RL inference on two large, well-known datasets exceeding a billion triples: LUBM10K and BTC2012. The LUBM10K dataset represents an optimistic case, meaning that if performance is poor with LUBM10K, then it will likely be poor on many datasets. On the other hand, the BTC2012 dataset represents a pessimistic case, meaning that if performance is good with BTC2012, then it is likely that performance will be good with other datasets. While the usual scalability metrics are used (speedup, efficiency, etc.), the Karp-Flatt metric reveals that inference is almost entirely parallel for LUBM10K data, demonstrating the practical feasibility of the theoretical findings. However, for BTC2012, it must be ensured that there is sufficient memory and load-balancing to achieve this high level of scalability on distributed memory architectures. Regardless, for feasible cases, very low times are achieved for LUBM10K (seconds) and BTC2012 (minutes).

Facebook's Graph API is an API for accessing objects and connections in Facebook's social graph. To give some idea of the enormity of the social graph underlying Facebook, it was recently announced that Facebook has 901 million users, and the social graph consists of many types beyond just users. Until recently, the Graph API provided data to applications in only a JSON format. In 2011, an effort was undertaken to provide the same data in a semantically-enriched, RDF format containing Linked Data URIs. This was achieved by implementing a flexible and robust translation of the JSON output to a Turtle output. This paper describes the associated design decisions, the resulting Linked Data for objects in the social graph, and known issues.

Science is increasingly motivated by the need to process larger quantities of data. It is facing severe challenges in data collection, management, and processing, so much so that the computational demands of "data scaling" are competing with, and in many fields surpassing, the traditional objective of decreasing processing time. Example domains with large datasets include astronomy, biology, genomics, climate/weather, and material sciences. This paper presents a real-world use case in which we wish to answer queries provided by domain scientists in order to facilitate discovery of relevant science resources. The problem is that the metadata for these science resources is very large and is growing quickly, rapidly increasing the need for a data scaling solution. We propose a system -- SGEM -- designed for answering graph-based queries over large datasets on cluster architectures, and we report early results for our current capability.

In web-scale semantic data analytics there is a great need for methods which aggregate uncertainty claims, on the one hand respecting the information provided as accurately as possible, while on the other still being tractable. Traditional statistical methods are more robust, but only represent distributional, additive uncertainty. Generalized information theory methods, including fuzzy systems and Dempster-Shafer (DS) evidence theory, represent multiple forms of uncertainty, but are computationally and methodologically difficult. We require methods which provide an effective balance between the complete representation of the full complexity of uncertainty claims in their interaction, while satisfying the needs of both computational complexity and human cognition. Here we build on J{\o}sang's subjective logic to posit methods in focused belief measures (FBMs), where a full DS structure is focused to a single event. The resulting ternary logical structure is posited to be able to capture the minimal amount of generalized complexity needed at a maximum of computational efficiency. We demonstrate the efficacy of this approach in a web ingest experiment over the 2012 Billion Triple dataset from the Semantic Web Challenge.

One of the greatest challenges facing computations on data crawled from the Web is the (in)ability to scale to such large quantities of data. While some computations are less challenged by this than others, inference on the Semantic Web is certainly limited in this regard. Parallelism has been employed to scale inference to larger datasets, but evaluations of recent works have fallen back on common parallel computing metrics that do not apply to this specific scalability challenge. In this position paper, the name data scaling is given to this scalability challenge, and the metric growth efficiency is defined.

As SPARQL endpoints are increasingly used to serve linked data, their ability to scale becomes crucial. Although much work has been done to improve query evaluation, little has been done to take advantage of caching. Effective solutions for caching query results can improve scalability by reducing latency, network IO, and CPU overhead. We show that simple augmentation of the database indexes found in common SPARQL implementations can directly lead to effective caching at the HTTP protocol level. Using tests from the Berlin SPARQL benchmark, we evaluate the potential of such caching to improve overall efficiency of SPARQL query evaluation.

The amount of RDF data published to the web is rapidly growing which has led to an increase in research of systems for handling such vast amounts of data. Employing parallelism has been a common approach, for which parallel I/O of RDF data can be very time-consuming. To reduce I/O load without requiring preprocessing, we propose a syntactic subset of the Turtle syntax called Sterno which is amenable for parallel I/O. We also evaluate the performance gain of using LZO compression, assuming preprocessing of data is allowable in a given use case. Our evaluation shows that Sterno documents improve on our previous work in parallel reading of N-triples documents, and LZO compression significantly reduces document size and read time.

dReasoning is a key element of the Semantic Web. For the Semantic Web to scale, it is required that reasoning also scales. This chapter focuses on two approaches to achieve this: The first deals with increasing the computational power available for a given task by harnessing distributed resources. These distributed resources refer to peer-to-peer networks, federated data stores, or cluster-based computing. The second deals with containing the set of axioms that need to be considered for a given task. This can be achieved by using intelligent selection strategies and limiting the scope of statements. The former is exemplified by methods substituting expensive web-scale reasoning with the cheaper application of heuristics while the latter by methods to control the quality of the provided axioms. Finally, future issues concerning information centralization and logics vs information retrieval-based methods, metrics, and benchmarking are considered.