Quality, trust, and utility of scientific data on the web: towards a joint model.

ABSTRACTIn science, quality is paramount. As scientists increasingly look to the Web to share and discover scientific data, there is a growing need to support the scientist in assessing the quality of that data. However, quality is an ambiguous and overloaded term. In order to support the scientific user in discovering useful data we have systematically examined the nature of "quality" by exploiting three, prevalent properties of scientific data sets: (1) that data quality is commonly defined objectively; (2) the provenance and lineage in its production has a well understood role; and (3) "fitness-for-use" is a definition of utility rather than quality or trust, where the quality and trust-worthiness of the data and the entities that produced that data inform its utility. Our study is presented in two stages. First we review existing information quality dimensions and detail an assessment-oriented classification. We introduce definitions for quality, trust and utility in terms of the entities required in their assessment; producer, provider, consumer, process, artifact and quality standard. Next we detail a novel and experimental approach to assessment by modelling the causal relationships between quality, trust, and utility dimensions through the construction of decision networks informed by provenance graphs. To ground and motivate our discussion throughout we draw on the European Bioinformatics Institute's Gene Ontology Annotations database. We present an initial demonstration of our approach with an example for ranking results from the Gene Ontology Annotation database using an emerging objective quality measure, the Gene Ontology Annotation Quality score.