In this paper we discuss the implementation of user-defined views over multihierarchical document-centric XML documents

Image-based electronic editions enable researchers to view and study in an electronic environment historical manuscript images intricately linked to edition, transcript, glossary and apparatus files. Building image-based electronic editions poses a two-fold challenge. For humanities scholars, it is important to be able to use image and text to successfully encode the desired features of the manuscripts. Computer Scientists must find mechanisms for representing markup in its association both with the images, text and other auxiliary files and for making the representation available for efficient querying. This paper addresses the architecture of one such solution, that uses efficient data structures to store image-based encodings in main memory and on disk.

Text has a non-hierarchical structure. Not surprisingly, searching for information in the content of a document often yieldsresults that overlap the structure within the document. It is often of a great interest to relate such results to the embedded document structure. In this work we present an extension of the XQuery language over multihierarchical document-centric XML documents. We illustrate the benefits of using multihierarchical XQuery for text-and-structure searches in document-centric XML documents More specifically, multihierarchical XQuery allows representing relationships between textsearch results and document structure even for cases wheresuch search results overlap markup boundaries and even incases when only one markup hierarchy is considered.

Document-centric XML document creation is a process of marking up textual content rather than typing text in a predefined structure. It turns out that, although the final document has to be valid with respect to the DTD/Schema used for the encoding, the "in-progress" document is almost never valid. At the same time, it is important to ensure that at each moment of time, the editor is working with an XML document that can be enriched with further markup to become valid. In this paper we explain the notion of potential validity of XML documents, which allows us to distinguish between XML documents that are invalid because the encoding is incomplete and XML documents that are invalid and no further encoding will make the document valid. We show that the set of potentially valid XML documents with respect to any DTD is context-free and we give a linear-time algorithm for checking potential validity for documents and document updates.

Documents have, in general, a multihierarchical structure (such as physical organization in the form of pages and lines, content organization in the form of paragraphs and sentences, etc.). Searching multihierarchical XML encoding presents a number of unique challenges for both computer scientists and document experts. We present an extension of the XQuery language suitable for searching multihierarchical XML

Management of multihierarchical XML encodings has attracted attention of a number of researchers both in databases [8] and in humanities[10]. Encoding documents using multiple hierarchies can yield overlapping markup. Previously proposed solutions to management of document-centric XML with overlapping markup rely on the XML expertise of humans and their ability to maintain correct schemas for complex markup languages.We demonstrate a unified solution for management of complex, multihierarchical document-centric XML. Our framework includes software for storing, parsing, in-memory access, editing and querying, multihierarchical XML documents with conflicting structures.

In recent years it has been argued that when XML encodings be- come complex, DOM trees are no longer adequate for query pro- cessing. Alternative representations of XML documents, such as multi-colored trees (7) have been proposed as a replacement for DOM trees for complex markup. In this paper we consider the use of Generalized Ordered-Descendant Directed Acyclic Graphs (GODDAGs) for the purpose of storing and querying complex do- cument-centric XML. GODDAGs are designed to store multihier- archical XML markup over the shared PCDATA content. They sup- port representation of overlapping markup, which otherwise can- not be represented easily in DOM. We describe how the seman- tics of XPath axes can be modified to define path expressions over GODDAG, and enhance it with the facilities to traverse and query overlapping markup. We provide efficient algorithms for axis eval- uation over GODDAG and describe the implementation of the query processor based on our definitions and algorithms.

We demonstrate the Edition Production Technology (EPT), an integrated development environment for building Image-based Electronic Editions (IBEE). EPT is developed in Java on top of Eclipse platform and benefits from the openness of Eclipse's plugin architecture and its portability (currently EPT runs on Windows XP, Linux, and Mac OS X). EPT provides software support for building image-based digital libraries of historic documents. Starting with high resolution images of manuscripts and transcriptions of them, EPT tools provide support for creating XML encoding of the electronic edition, searching the electronic edition, linking text and images, and publishing the electronic edition (using filters and XSLT)

To properly encode properties of textual documents using XML, multiple markup hierarchies must be used, often leading to conflicting markup in encodings. Text Encoding Initiative (TEI) Guidelines [1] recognize this problem and suggest a number of ways to incorporate multiple hierarchies in a single well-formed XML document. In this paper, we present a framework for processing XPath queries over multi-hierarchical XML documents represented using fragmentation, one of the TEI-suggested techniques. We define the semantics of XPath over DOM trees of fragmented XML, extend the path expression language to cover overlap in markup, and describe FragXPath, our implementation of the proposed XPath semantics over fragmented markup.

The problem of overlapping markup hierarchies, first mentioned in the context of SGML, often occurs in XML text encoding applications for humanities. Previous solutions to the problem rely on manual maintenance of the markup and address only the problem of representing overlapping features in XML, leaving the issues of automated maintenance and querying open. As a consequence, traditional XML tools are of little practical use when dealing with overlapping markup. In this work we demonstrate the implementation of our framework for management of concurrent XML hierarchies from a computer science perspective. We propose an underlying model, data structures, APIs, and algorithms so that the most of the burden of managing concurrent XML hierarchies would be born by the software.

The process of authoring document-centric XML documents in humanities disciplines is very different from the approach espoused by the standard XML editing software with the data-centric view of XML. Where data-centric XML is generated by first describing a tree structure of the encoding and then providing the content for the leaf elements, document-centric encodings start with content which is then marked up. In the paper we describe our approach to authoring document-centric XML documents and the tool, xTagger, originally developed for this purpose within the Electronic Boethius project [2], otherwise enhanced within the ARCHway project [5], an interdisciplinary project devoted to development of methods and software for preparation of image-based electronic editions of historic manuscripts.

XML, initially designed for large scale text publishing, has rapidly evolved as a standard for a wide variety of data exchange and representation applications. With an increased volume of data, XML data management has been the subject of intensive research. Database research groups have concentrated on building database management frameworks around semistructured data represented as XML. At the same time, humanities research groups have concentrated on development of application specific XML-compliant markup languages, and application of XML to encoding a wide array of documents. Two major kinds of XML documents emerge from applications: data-centric and document-centric. Data-centric documents are characterized by a fairly regular structure and occur as a standard format for data exchange and representation of semistructured data. Document-centric XML has, in general, a much more irregular structure and is often encountered as the means of document markup. In recent years, a number of applications of XML to document-centric encoding have led to markup that could not be stored in a hierarchical XML document (the concurrent markup hierarchies problem). This is mainly a consequence of the multi-hierarchical nature of text documents: the physical location hierarchy (document pages and lines), the text structure hierarchy (paragraphs, sentences, and words), etc. A prominent example of document-centric XML with multiple hierarchies is the XML encoding of manuscript folio images: the heterogeneous information to be encoded (from text and images) is very rarely hierarchical. The problem of concurrent markup hierarchies in document-centric XML encodings has attracted attention of a number of humanities researchers in recent years. Previously proposed solutions to this problem rely on the XML expertise of humans and their ability to maintain correct schemas for complex markup languages. This thesis introduces a framework that allows the humans to concentrate on the semantic aspects of the encoding, while leaving the burden of maintaining XML documents to the software. We formally define the notion of concurrent markup hierarchies and concurrent XML documents and we give algorithms for document-centric XML data management, with a special focus on document-centric XML documents with concurrent markup.

The process of creation of document-centric XML documents often starts with a prepared textual content, into which the editor introduces markup. In such situations, intermediate XML is almost never valid with respect to the DTD/Schema used for the encoding. At the same time, it is important to ensure that at each moment of time, the editor is working with an XML document that can enriched with further markup to become valid. In this paper we introduce the notion of potential validity of XML documents, which allows us to distinguish between XML documents that are invalid because the encoding is simply incomplete and XML documents that are invalid because some of the DTD rules guiding the structure of the encoding were violated during the markup process. We give a linear-time algorithm for checking potential validity for documents.