We explore the design and implementation of Rover, a postmortem debugger for XQuery. Rather than being based on the traditional breakpoint model, Rover acknowledges XQuery's nature as a functional language: the debugger follows a declarative debugging paradigm in which a user is enabled to observe the values of selected XQuery subexpressions. Rover has been designed to hook into Pathfinder, an XQuery compiler that emits relational algebra plans for evaluation on commodity relational database back-ends. The debugger instruments the subject query with fn:trace() calls which, at query runtime, populate database tables with relational representations of XQuery item sequences. Thanks to Pathfinder's loop-lifting compilation strategy, a Rover trace (1) may span multiple XQuery for iteration scopes and (2) allows for interactive debugging sessions that can arbitrarily replay iterations in a unique forward/backward fashion. Since the query runtime as well as the debugger are database-supported, Rover is scalable and supports the observation of very data-intensive XQuery expressions.
XQuery processors that load the input into main memory suffer from huge memory demands. Yet for the evaluation of many queries, large parts of the input are actually irrelevant. In XML document projection, this data is recognized and not loaded in the first place. However, there are also queries where little can be gained by projection. We have observed that these queries tend to require large parts of the input only for generating output. This suggests that such "bulk" data may be stored and treated differently from data that is actually traversed in query evaluation. In this paper, we present a technique to recognize bulk data while loading XML documents for the evaluation of composition-free XQuery. Our approach is coupled with XML document projection, and utilizes a finite automaton that is expressly suited for matching path expressions. We show in an exploratory analysis that bulk data arises in practice, and discuss ongoing work along the line of bulk-bypassing in main memory-based XQuery engines.
We address the problem of selection of fragments of XML input data as required for XQuery evaluation. Rather than first selecting individual fragments in isolation and then bringing them together as required by multiple variable bindings, we select the tuples of co-related fragments at once in one-pass over the input. Our approach is event-driven and correspondingly does not require building the input data in memory. The tuples as needed for generating the output are reported as early as possible. Combined with an incremental garbage collection scheme of the buffers, our approach allows to store at any time only as much data as necessarily needed for the query evaluation.
We present DXQ, an extension of XQuery to support the effective and efficient development of distributed XML applications. A DXQ program can invoke remote DXQ programs both synchronously and asynchronously and can dynamically ship DXQ code to execute at remote servers. We illustrate the power of the language with two distributed applications: the resolution algorithm of the Domain Name System (DNS) and the Narada overlay-network protocol. Our implementation permits concurrent evaluation of DXQ expressions at each server and can produce results extensionally (as XML values) or intensionally (as DXQ expressions).
We discuss query evaluation for XML-based server systems where the same query is evaluated on every incoming XML message. In a typical scenario, many of the incoming messages will be highly similar to each other. Current XML query evaluators reevaluate the query from scratch on every message. We call substructures that occur in many input documents template fragments, and introduce a novel template folding method that allows to move the work of evaluating the query on recurring document substructures from the query execution engine into the query compiler. Similar to constant folding, our method avoids run-time evaluation of intermediate results whose value only depends on information that is already available at compile time. For XPath location paths, we propose a representation for such invariant intermediate results, and show how it can be incorporated into query execution plans. Such augmented execution plans improve query performance when evaluating the same query on subsequent input documents.
paper proposes an extension to the XQuery language to solve the problem of positional grouping: that is, problems in which it is necessary to convert a flat sequence into a hierarchy by recognizing patterns in the sequence of items. Positional grouping is contrasted with value-based grouping, where the allocation of items to groups is based on common values rather than on the positional relationships of the items in the sequence. The approach is based on analyzing a set of use cases, derived from real-world experience.
XQuery is a declarative language for querying and updating XML data sources. Interfacing XQuery to a host programming language is difficult because of the type system mismatch, and global optimization is difficult in a mixed-language environment. In this paper, we investigate a small extension called XQueryP that enables XQuery expressions to exchange state information through variables. This extension makes it easier to develop applications in XQuery without relying on a host programming language. We draw an analogy between the proposed extension and similar extensions that have been added over the years to the SQL query language for similar reasons.
Text has a non-hierarchical structure. Not surprisingly, searching for information in the content of a document often yieldsresults that overlap the structure within the document. It is often of a great interest to relate such results to the embedded document structure. In this work we present an extension of the XQuery language over multihierarchical document-centric XML documents. We illustrate the benefits of using multihierarchical XQuery for text-and-structure searches in document-centric XML documents More specifically, multihierarchical XQuery allows representing relationships between textsearch results and document structure even for cases wheresuch search results overlap markup boundaries and even incases when only one markup hierarchy is considered.
We present a high-performance, pull-based streaming processor for XQuery, called XQPull, that can handle many essential features of the language, including general predicates, recursive queries, backward axis steps, and function calls, using a very small amount of caching. Our framework is based on a new type of event streams, called retarded streams, which allow multiple and nested streams to be interleaved in the same physical stream, while postponing the caching of input events until is absolutely necessary, typically at the end of the query evaluation, just before the results are ready to print.
XML annotations are a widely occurring phenomenon in many application fields, and XML databases should be used to store and query such data. To provide intuitive and fast querying of annotations, we make a case for extending XPath with four new axis steps, that correspond with socalled StandOff joins, introduced here. The new steps can be efficiently implemented using a region index and fast looplifted StandOff MergeJoin algorithms. These techniques were added to the open-source XML DBMS MonetDB/XQuery, and we show in our evaluation it thus becomes capable of interactively querying >GB annotation databases.
We present in this paper a system for rewriting user-specific XPath queries for higher sharing of common sub-expressions in a streaming environment. We rewrite these queries according to an extracted schema from a pre-processed stream and we apply the rewritten queries to a subsequent stream. We show that this rewriting yields a much higher throughput while keeping an error rate under control.
Markup languages, representations, schemas, and tools have significantly increased the ability for organizations to share their information. Languages such as the Extensible Markup Language (XML) provide a vehicle for organizations to represent information in a common, machine-interpretable format. Furthermore languages, such as the Document Type Definition Language (DTD) and XML Schema Definition Language (XSD) allow organizations to share the schema and structure of their data. Though these approaches facilitate the collaboration and integration of inter-organizational information, the reality is that the schema languages are reasonably difficult to learn, and automated schema integration (without semantics or ontology mappings) is currently an open problem. We introduce an architecture to facilitate organizational collaboration. In this paper, we introduce such an architecture that combines the push features of the publish/subscribe protocol with distributed registry capabilities. In addition, a Java-based, service-oriented implementation entitled Sharx is described and evaluated.
As XML (cid:12)nds its place in information technology, query languages for XML attract much attention. Among them XPath is the most known. This article reveals a surprising fact that the upcoming XPath 2.0 is capable of sorting sequences, which is proved formally. We also mark out an incompleteness of XQuery Core and an insu(cid:14)ciency of definition of XQuery Core sorting semantics. The fact that XPath 2.0 can sort arbitrary sequences is used to (cid:12)x it.
We describe the architecture and some aspects of System R/X, a native XML database engine that is built on the same mature infrastructure for a relational database and integrated with the relational engine. We describe what parts of the infrastructure can be reused, what need to be extended, and what are totally new to the XML database and their techniques. Our overall strategy is to base XML storage and search on the scalable relational technology with substantial extensions. Many techniques are novel to our knowledge. We also provide perspectives along the discussion and point out some open research issues.
Nous etudions dans cette these le probleme de l'independance physique des donnees dans les bases de donnees XML. Dans une premiere partie de cette these nous proposons les modules d'acces XML (XML Access Modules ou XAMs) - un langage de motifs d'arbre concu pour exprimer un grand sous-ensemble de XQuery, et enrichi avec des noeuds optionnels (permettant de capturer des motifs qui couvrent plusieurs requetes imbriquees) et des identiants structurels (qui augmentent les possibilites de reecriture). Nous demontrons que ce langage des vues peut etre utilise pour decrire uniformement un grand nombre de schemas de stockage, d'index et de vues materialisees. Dans une deuxieme partie de cette these nous etudions le probleme de la reecriture des requetes XQuery a travers des vues exprimees par des modules d'acces XML. Dans un premier temps, nous presentons un algorithme capable d'extraire des motifs XAM a partir des requetes XQuery et nous demontrons l'importance d'utiliser des vues qui peuvent enjamber plusieurs blocs XQuery imbriques. Par la suite, nous etudions le probleme de la reecriture des requetes en utilisant des vues materialisees, ou la requete et les vues sont decrites par des vues XAM. Nous caracterisons la complexite de l'inclusion de motifs d'arbre et de la reecriture des requetes sous les contraintes exprimees par des resumes structurels, dont une forme augmentee permettra egalement d'exprimer des contraintes d'integrite.
Bell Laboratories Lucent Technologies benedikt,vyas @research.bell-labs.com Icar CNR Via P. Bucci 41C, 87036 Rende, Italy bonifati@icar.cnr.it D.E.I.S., University of Calabria Via P. Bucci 41C, 87036 Rende, Italy flesca@deis.unical.it The need to extend XQuery to support updates has been recognized both in the research and the standards community. Several language proposals and prototype implementations have been put forward, and update language requirements are being defined within the W3C. Most proposals center around the use of update primitives applied to XQuery expressions, along with a variant of the FLWR loop construct binding variables within a block of basic update statements. In defining a precise semantics for such statements a number of issues arise: one must decide how conflicts among updates are to be resolved, and how query evaluation interacts with update application. In this work we provide a framework for defining alternative semantics for updates, and identify within this space what is (thus far) the consensus choice: that semantics involves a twostage execution process, in which query evaluation is performed first, after which a generated sequence of concrete updates is applied in a fixed order determined by query output. This results in a clean deterministic semantics which facilitates analysis. A drawback is that the evaluation of the language can be inefficient. One would prefer to perform updates eagerly before further evaluation, or to re-order the update operations. We focus on an optimization of the “standard semantics”, in which updates are performed as soon as they are generated. We present a static analysis for determining when this optimization can be exploited. Experiments on the implementation of this analysis, implemented on top of Galax, show that the overhead is minimal.
