Dataspace management systems (DSMSs) hold the promise of pay-as-you-go data integration. We describe a comprehensive model of DSMS functionality using an algebraic style. We begin by characterizing a dataspace life cycle and highlighting opportunities for both automation and user-driven improvement techniques. Building on the observation that many of the techniques developed in model management are of use in data integration contexts as well, we briefly introduce the model management area and explain how previous work on both data integration and model management needs extending if the full dataspace life cycle is to be supported.We show that many model management operators already enable important functionalities (e.g., the merging of schemas, the composition of mappings, etc.) and formulate these capabilities in an algebraic structure, thereby giving rise to the notion of the core functionality of a DSMS as a many-sorted algebra. Given this view, we show how core tasks in the dataspace life cycle can be enacted by means of algebraic programs. An extended case study illustrates how such algebraic programs capture a challenging, practical scenario.

Dataspaces aim to remove upfront cost in the generation of the schema mappings that reconcile schematic heterogeneities, and to incrementally improve the generated mappings based on user feedback. The reconciliation of schematic heterogeneities is a crucial step for translating queries between a mediating schema and data sources. The generation of schema mappings depends on the elicitation of conceptually equivalent schema constructs and information on schematic heterogeneities. Furthermore, many dataspace operations manipulate associations between schemas, for example for generating a global schema to mediate user queries. With a view to minimizing upfront costs associated with understanding the relationships between schemas, many schema matching algorithms and tools have been developed for postulating equivalent schema constructs. However, they derive simple associations between schema constructs, and do not provide rich information on schematic heterogeneities. Without manual refinement, the elicitation of conceptually equivalent schema constructs and schematic heterogeneities may create uncertainties that must be managed.The schematic correspondences captures a wide range of one-to-one and many-to-many schematic heterogeneities. This thesis investigates the use of schematic correspondences as a central component in a dataspace management system. To support query evaluation in a dataspace in which relationships between schemas are represented using schematic correspondences, we propose a mechanism for automatically generating schema mappings from the schematic correspondences. We then characterise model management operators, which can underpin the bootstraping and maintenance of dataspaces, over schematic correspondences. To support the management of uncertainty in dataspaces, we propose techniques for quantifying uncertainty in the equivalence of schema constructs from evidence in the form of similarity scores and user feedback, and provide a flexible framework for incrementally updating the uncertainties in the light of new evidence.

The vision of dataspaces is to provide various of the benefits of classical data integration, but with reduced up-front costs. Combining this with opportunities for incremental refinement enables a ‘pay-as-you-go' approach to data integration, resulting in simplified integrated access to distributed data. It has been speculated that model management could provide the basis for Dataspace Management, however, this has not been investigated until now. Here, we present DSToolkit, the first dataspace management system that is based on model management, and therefore, benefits from the flexibility provided by the approach for the management of schemas represented in heterogeneous models, supports the complete dataspace lifecycle, which includes automatic initialisation, maintenance and improvement of a dataspace, and allows the user to provide feedback by annotating result tuples returned as a result of queries the user has posed. The user feedback gathered is utilised for improvement by annotating, selecting and refining mappings. Without the need for additional feedback on a new data source, these techniques can also be applied to determine its perceived quality with respect to already gathered feedback and to identify the best mappings over all sources including the new one.

The vision of dataspaces proposes an alternative to classical data integration approaches with reduced up-front costs followed by incremental improvement on a pay-as-you-go basis. In this paper, we demonstrate DSToolkit, a system that allows users to provide feedback on results of queries posed over an integration schema. Such feedback is then used to annotate the mappings with their respective precision and recall. The system then allows a user to state the expected levels of precision (or recall) that the query results should exhibit and, in order to produce those results, the system selects those mappings that are predicted to meet the stated constraints.

The vision of dataspaces has been articulated as providing various of the benefits of classical data integration but with reduced up-front costs, which, combined with opportunities for incremental refinement, enables a “pay as you go” approach to the data integration problem. However, results that seek to realise the vision tend to make design commitments, often to meet quite specific application assumptions, that are likely to restrict their wider use. Instead of precommitting to a specific solution, we build on research in model management and present a generic framework consisting of a collection of types and operations for dataspace management systems that can be instantiated in various ways. The key extension for dataspaces is the integration of user feedback as annotations to model management constructs, and the development of operations that take account of these annotations. The flexibility of the framework is demonstrated through various case studies that meet differing requirements.

Mapping specification has been recognised as a critical bottleneck to the large scale deployment of data integration systems. A mapping is a description using which data structured under one schema are transformed into data structured under a different schema, and is central to data integration and data exchange systems. In this paper, we argue that the classical approach of correspondence identification followed by (manual) mapping generation can be simplified through the removal of the second step by judicious refinement of the correspondences captured. As a step in this direction, we present in this paper a model for schematic correspondences that builds on and extends the classification proposed by Kim et al. to cater for the automatic derivation of mappings, and present an algorithm that shows how correspondences specified in the model proposed can be used for deriving schema mappings. The approach is illustrated using a case study from integration in proteomics.

At the present time, data management systems need to support the explosive growth in information scale. Data managed within such systems are shared and stored across a number of heterogeneous sources, some of which are created, modelled and maintained independently by different personnel and in different ways. We call this a dataspace, in other words, a space of heterogeneous data sources. Since there does not exist an universal standard to restrict the definition of schema that describes the structure of data from a particular domain (or with the same meaning), schematic heterogeneity could appear in various aspects. In other words, two schemas can represent the information from the same domain concept either use identical or conflicting schematic representations, this is called schematic correspondences. It can exist at the entity level; for example, one schema at a certain data source can assign an entity type named undergraduate-student to represent the concept: all the undergraduate students of a particular institution. By contrast, a schema at a different data source can assign either an identically named entity type, or a differently named entity type, for example with a shorter name undergraduate, to represent the same concept. Schematic correspondences can also exist at the attribute level; for example, an undergraduate student can be described by his first-name and last-name in one data source and by full-name and used-name in another data source. One of the reasons that data management care about heterogeneity among data sources is because data can be shared and not restricted at a specific source. When user posing a query on a particular data source, his interests are not just in what information he can be retrieved from that data source, but also from other sources which can provide semantically equivalent answers for that query. Think about what web search engines like Google can serve us: they allow a user to pose a keyword on a pool of web information, and returns a set of ranked web pages with the most relevant ones on the top of the rank. However, not every time search engine can precisely identify what are the most relevant answers to match the user’s desire, since it can not fully understand, and be provided with information on, semantic relationships among heterogeneous information. The eventual goal of data sharing and data integration is that we can get everything we want from everything there is, and the ’everything’ we actually get precisely match ’everything’ we should idealy get. To meet this goal, we need to know the semantic relationships between heterogeneous data sources, as well as in what ways that they correspond to each other.