Methodologies for virtual examination of Cultural Heritage artifacts through Reflectance Transformation Imaging (RTI) are gaining interest. Although at the beginning this techniques were designed to aid Cultural Heritage specialists in the inspection and interpretation process, the recent advances of 3D web visualization platforms are increasing our capability to open this type of visual inspection to the ordinary public. We present the design and implementation of a system that provide the access of ordinary people (visitors of the museum or web surfers) to a collection of ancient coins, preserved in the San Matteo Museum of Pisa (Italy). Those coins have been digitized to produce RTI representations and are presented to museum visitors together with some data telling their story and explaining their value. The interactive presentation system, designed to be deployed either by a museum kiosk or by a web site, is composed of two integrated sections: a brief introduction to the different subsets of coins; an  interactive visualization of the RTI representation of each coin, by means of a specific RTI viewer. The coins are organized in different (possibly overlapping) subsets,  to allow us to better present them to a public of inexperienced visitors. Starting from the main theme, the Historical Collection  of the coins, we have some subsets about the iconography, the different production techniques (coining or fusion), the value of the coins, etc. After the selection of a specific subset, the user is presented a basic page containing an hypertext description with some basic information, a draggable image gallery with the thumbnails of the coins in the subset and the RTI viewer to display the specific selected coin.  The viewer supports visualization and interaction with the RTI image through a basic set of functionalities: changing the light direction, zooming, panning over the image, flipping the coin to switch between the two sides and finally the possibility to show some hot spots, which link specific locations/areas on the coin with descriptive data (presented by means of HTML hypertext page).

Scientific investigations of artworks are crucial in terms of preservation since they provide a measurable evaluation of the materials and the state of conservation. This is the case of Antonello da Messina’s painting “Ecce Homo”: its delicate state of conservation, with the need for constant monitoring, required a broad and in-depth diagnostic campaign to support the restorers. The project was carried out entirely in situ using non-invasive cutting-edge techniques and proposes a multimodal and data-centric approach, integrating 3D and 2D methodologies. The surface irregularities and the support were analysed with a structured-light 3D scanner and X-ray tomography. The painting materials were investigated with X-ray fluorescence scanning (MA-XRF) and reflectance hyperspectral imaging (HSI). Primarily, the data were jointly used for a scientific scope and provided new knowledge of the painting in terms of materials and painting techniques. In addition, two web-based interactive platforms were developed: one to provide restorers and experts with a new perspective of the hidden geometries of the painting, and the other targeted at the general public for dissemination purposes. The results of the Ecce Homo scientific analysis were exhibited, using a touch-screen interface, and developed for different user levels, from adults to kids.

The Cultural Heritage (CH) domain is a field where many different visual media are constituent elements of the main activities: study, conservation, dissemination, and presentation to the public (museum visitors, tourists, practitioners). Those media are usually used in isolation, adopting specific visualization tools. This paper aims to present several experiences where multiple visual media have been used in a coordinated manner by fusing or presenting them in the same visualization context. These approaches experimented with new interaction and visualization methodologies to use different media in a synergic way. CH domain is an ideal field of experimentation of the potential of media integration/fusion/ cross-analysis. According to our understanding, using multiple media can improve insight capability. We guide the reader in the analysis of some pioneering experiences and approaches and try to deduce, for each of them, the potential improvement granted in terms of data communication or analysis. A final discussion tries to highlight the work needed for a wider acceptance and increased impact of those approaches.

Annotation support in interactive systems is often considered a simple task by the CG community, since it entails the apparently easy selection of a region and its connection with some information. The reality appears more complex. The scope of this paper is two‐fold: first, to review the status of this domain, discussing and characterizing several approaches proposed in literature to manage annotations over geometric models; second, to present in detail an innovative solution proposed and assessed in the framework of Cultural Heritage (CH) applications, called ClippingVolumes. At the annotation definition stage ClippingVolumes uses 3D data to characterize the annotation region; subsequently, annotations are visualized by adopting a two‐pass rendering solution which uses stencil buffers, thus without introducing new geometric elements, changing the topology or duplicating geometry elements. It solves most of the issues that afflict the current state of the art, such as fragmentation, annotation transfer to multiple representations and multi‐resolution data encoding. The latter is a mandatory requirement to produce efficient web‐based systems. We implemented and we fully tested this approach in the framework of a complex system that supports the documentation of CH restoration projects.

Pottery is of fundamental importance for understanding archaeological contexts, facilitating the understanding of production, trade flows, and social interactions. Pottery characterisation and the classification of ceramics is still a manual process, reliant on analogue catalogues created by specialists, held in archives and libraries. The ArchAIDE project worked to streamline, optimise and economise the mundane aspects of these processes, using the latest automatic image recognition technology, while retaining key decision points necessary to create trusted results. Specifically, ArchAIDE worked to support classification and interpretation work (during both fieldwork and post-excavation analysis) with an innovative app for tablets and smartphones. This article summarises the work of this three-year project, funded by the European Union's Horizon 2020 Research and Innovation Programme under grant agreement N.693548, with a consortium of partners representing both the academic and industry-led ICT (Information and Communications Technology) domains, and the academic and development-led archaeology domains. The collaborative work of the archaeological and technical partners created a pipeline where potsherds are photographed, their characteristics compared against a trained neural network, and the results returned with suggested matches from a comparative collection with typical pottery types and characteristics. Once the correct type is identified, all relevant information for that type is linked to the new sherd and stored within a database that can be shared online. ArchAIDE integrated a variety of novel and best-practice approaches, both in the creation of the app, and the communication of the project to a range of stakeholders.

Close-Range Photogrammetry (CRP) and Reflectance Transformation Imaging (RTI) are two of the most used image-based techniques when documenting and analyzing Cultural Heritage (CH) objects. Nevertheless, their potential impact in supporting study and analysis of conservation status of CH assets is reduced as they remain mostly applied and analyzed separately. This is mostly because we miss easy-to-use tools for of a spatial registration of multimodal data and features for joint visualisation gaps. The aim of this paper is to describe a complete framework for an effective data fusion and to present a user friendly viewer enabling the joint visual analysis of 2D/3D data and RTI images. This contribution is framed by the on-going implementation of automatic multimodal registration (3D, 2D RGB and RTI) into a collaborative web platform (AIOLI) enabling the management of hybrid representations through an intuitive visualization framework and also supporting semantic enrichment through spatialized 2D/3D annotations.

Terrestrial laser scanning campaigns provide an important means to document the 3D structure of historical sites. Unfortunately, the process of converting the 3D point clouds acquired by the laser scanner into a coherent and accurate 3D model has many stages and is not generally automated. In particular, the initial cleaning stage of the pipeline—in which undesired scene points are deleted—remains largely manual and is usually labour intensive. In this article, we introduce a semi-automated cleaning approach that incrementally trains a random forest (RF) classifier on an initial keep/discard point labelling generated by the user when cleaning the first scan(s). The classifier is then used to predict the labelling of the next scan in the sequence. Before this classification is presented to the user, a denoising post-process, based on the 2D range map representation of the laser scan, is applied. This significantly reduces small isolated point clusters that the user would otherwise have to fix. The user then selects the remaining incorrectly labelled points and these are weighted, based on a confidence estimate, and fed back into the classifier to retrain it for the next scan. Our experiments, across 8 scanning campaigns, show that when the scan campaign is coherent, i.e., it does not contain widely disparate or contradictory data, the classifier yields a keep/discard labelling that typically ranges between 95% and 99%. This is somewhat surprising, given that the data in each class can represent many object types, such as a tree, person, wall, and so on, and that no further effort beyond the point labeling of keep/discard is required of the user. We conducted an informal timing experiment over a 15-scan campaign, which compared the processing time required by our software, without user interaction (point label correction) time, against the time taken by an expert user to completely clean all scans. The expert user required 95mins to complete all cleaning. The average time required by the expert to clean a single scan was 6.3mins. Even with current unoptimized code, our system was able to generate keep/discard labels for all scans, with 98% (average) accuracy, in 75mins. This leaves as much as 20mins for the user input required to relabel the 2% of mispredicted points across the set of scans before the full system time would match the expert’s cleaning time.

We present a method for turning a flash selfie taken with a smartphone into a photograph as if it was taken in a studio setting with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in an ad-hoc acquisition campaign. Each pair consists of one photograph of a subject’s face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend defects introduced by a close-up camera flash, such as specular highlights, shadows, skin shine, and flattened images.

3D scanning is an effective technology for dealing at different levels the state of conservation/deformation of a panel painting, from the micro-geometry of the craquelure to the macro-geometry of the supported used. Unfortunately, the current solutions used to analyze multiple 3D scans acquired over time are based on very controlled acquisition procedures, such as the use of target reference points that are stationary over time and fixed to the artwork, or on complex hardware setups to keep the acquisition device fixed to the artwork. These procedures are challenging when a long monitoring period is involved or during restoration when the painting may be moved several times. This paper presents a new and robust approach to observe and quantify the panel deformations of artworks by comparing 3D models acquired with different scanning devices at different times. The procedure is based on a non-rigid registration algorithm that deforms one 3D model over the other in a controlled way, extracting the real deformation field. We apply the method to the 3D scanning data of the unfinished panel painting “Adorazione dei Magi” by Leonardo da Vinci. The data were acquired in 2002 and 2015. First, we analyze the two 3D models with the classical distance from the ideal flat plane of the painting. Then we study the type of deformation of each plank of the support by fitting a quadric surface. Finally, we compare the models before and after the deformation computed by a non-rigid registration algorithm. This last comparison enables the panel deformation to be separated from the structural changes (e.g. the structural restorations on the back and the missing pieces) of the artwork in a more robust way.

Through this White Paper, which gathers contributions from experts of 3D data as well as professionals concerned with the interoperability and sustainability of 3D research data, the PARTHENOS project aims at highlighting some of the current issues they have to face, with possible specific points according to the discipline, and potential practices and methodologies to deal with these issues. During the workshop, several tools to deal with these issues have been introduced and confronted with the participants experiences, this White Paper now intends to go further by also integrating participants feedbacks and suggestions of potential improvements. Therefore, even if the focus is put on specific tools, the main goal is to contribute to the development of standardized good practices related to the sharing, publication, storage and long-term preservation of 3D data.

Visual sampling techniques represent a valuable resource for a rapid, non-invasive data acquisition for underwater monitoring purposes. Long-term monitoring projects usually requires the collection of large quantities of data, and the visual analysis of a human expert operator remains, in this context, a very time consuming task. It has been estimated that only the 1-2 % of the acquired images are later analyzed by scientists (Beijbom et al., 2012). Strategies for the automatic recognition of benthic communities are required to effectively exploit all the information contained in visual data. Supervised learning methods, the most promising classification techniques in this field, are commonly affected by two recurring issues: the wide diversity of marine organism, and the small amount of labeled data. In this work, we discuss the advantages offered by the use of annotated high resolution ortho-mosaics of seabed to classify and segment the investigated specimens, and we suggest several strategies to obtain a considerable per-pixel classification performance although the use of a reduced training dataset composed by a single ortho-mosaic. The proposed methodology can be applied to a large number of different species, making the procedure of marine organism identification an highly adaptable task.

This chapter studies how, in establishing Europe's boundaries, the Congress of Vienna ratified emancipation's division into the three regions of western, central, and eastern Europe. The Congress affected the status of Jews in western Europe only in bringing equality to the Jews in Belgium. In eastern Europe, the Congress brought territorial adjustments; the larger legal arrangements remained in place. In contrast, in central Europe, the Congress failed to bring unification and a uniform law for Jews. The German states became a mosaic of disparate laws. Many polities revived the ancien régime. Moreover, the dualism of “state” versus “local” or municipal rights began to emerge. Italy offered a similar profile of multiple states, disparate legal statuses, and in many states a resuscitated ancien régime.

This chapter studies how, in establishing Europe's boundaries, the Congress of Vienna ratified emancipation's division into the three regions of western, central, and eastern Europe. The Congress affected the status of Jews in western Europe only in bringing equality to the Jews in Belgium. In eastern Europe, the Congress brought territorial adjustments; the larger legal arrangements remained in place. In contrast, in central Europe, the Congress failed to bring unification and a uniform law for Jews. The German states became a mosaic of disparate laws. Many polities revived the ancien régime. Moreover, the dualism of “state” versus “local” or municipal rights began to emerge. Italy offered a similar profile of multiple states, disparate legal statuses, and in many states a resuscitated ancien régime.

Relightable images have been widely used as a valuable tool in Cultural Heritage (CH) artifacts, including coins, bas-reliefs, paintings, and epigraphs. Reflection Transformation Imaging (RTI), a commonly used type of relightable images, consists of a per-pixel function which encodes the reflection behavior, estimated from a set of digital photographs acquired from a fixed view. Web visualisation tools for RTI images currently require to transmit substantial quantities of data in order to achieve high fidelity renderings. We propose a web-friendly compact representation for RTI images based on a joint interpolation-compression scheme that combines a PCA-based data reduction with a Gaussian Radial Basis Function (RBF) interpolation exhibiting superior performance in terms of quality/size ratio. This approach can be adapted also to other data interpolation schemes, and it is not limited to Gaussian RBF. The rendering part is simple to implement and computationally efficient allowing real-time rendering on low-end devices.

This study presents a practical method to estimate dimensions of Paramuricea clavata colonies using generic photographic datasets collected across wide areas. Paramuricea clavata is a non-rigid, tree-like octocoral; this morphology greatly affects the quality of the sea fans multi-view stereo matching reconstruction, resulting in hazy and incoherent clouds, full of “false” points with random orientation. Therefore, the standard procedure to take measurements over a reconstructed textured surface in 3D space is impractical. Our method overcomes this problem by using quasi-orthorectified images, produced by projecting registered photos on the plane that best fits the point cloud of the colony. The assessments of the measures collected have been performed comparing ground truth data set and time series images of the same set of colonies. The measurement errors fall below the requirements for this type of ecological observations. Compared to previous works, the presented method does not require a detailed reconstruction of individual colonies, but relies on a global multi-view stereo reconstruction performed through a comprehensive photographic coverage of the area of interest, using a lowcost pre-calibrated camera. This approach drastically reduces the time spent working on the field, helping practitioners and scientists in improving efficiency and accuracy in their monitoring plans.

This paper presents our digital publishing process and its editing system developed for a specific use case - the scholarly digital edition of a set of Roman inscribed lead tags - but suited for analyzing a broad variety of text-bearing objects. This pipeline covers the spectrum of the activities from data acquisition of the visual aspects to complex analysis of the writings and text of incised inscriptions. It employs a two-dimensional medium, Reflection Transformation Images (RTI), to support visual analysis. The visual and textual data it generates are open access, ready to be used for machine learning processes.

Relightable images have demonstrated to be a valuable tool for the study and the analysis of coins, bas-relief, paintings, and epigraphy in the Cultural Heritage (CH) field. Reflection Transformation Imaging (RTI) are the most diffuse type of relightable images. An RTI image consists in a per-pixel function which encodes the reflection behavior, estimated from a set of digital photographs acquired from a fixed view. Even if web visualization tools for RTI images are available, high fidelity of the relighted images still requires a high amount of data to be transmitted. To overcome this limit, we propose a web-friendly compact representation for RTI images which allows very high quality of the rendered images with a relatively small amount of data required (in the order of 6-9 standard JPEG color images). The proposed approach is based on a joint interpolation-compression scheme that combines a PCA-based data reduction with a Gaussian Radial Basis Function (RBF) interpolation. We will see that the proposed approach can be adapted also to other data interpolation schemes, and it is not limited to Gaussian RBF. The proposed approach has been compared with several techniques, demonstrating its superior performance in terms of quality/size ratio. Additionally, the rendering part is simple to implement and very efficient in terms of computational cost. This allows real-time rendering also on low-end devices.