In this article, the so-called Quill Feature method is tested as a means for scribal attribution in mediaeval manuscripts and placed in a palaeographic context. The Quill Features encompass a range of measurable features such as stroke curvature and variation in thewidth of strokes that graphs are comprised of, giving consideration to the angle at which they are drawn. The construction of the mediaeval pen, the quill, creates variation in the width of strokes, most importantly due to direction and force of pressure. It has been assumed that individual patterns can be discerned in this variation. The manuscript chosen for investigation is Cod. U ps. C 61, which contains a number of S t B irgitta’s R evelations and other material connected with this saint. Earlier researchers claimed that three or four scribes produced this manuscript, and this assertion was tested with the Quill Feature method. It transpired that the resultant values, calculated through measuring the investigated features, were very similar for the pages produced by the same scribe, which means that the method identifies important regularities in the scribal hands. Nevertheless, some of the investigated hands showed great similarities and were difficult for the computer to separate. Finally, the strengths and weaknesses of the digital methods of script analysis are discussed. The main advantage of these approaches is that a computer can measure characteristics of the script that cannot be discernedwith the naked eye, and with a high degree of exactness. One weakness is that the measurements obtained are given in numbers, and can thus be difficult to apply to the script signs. Furthermore, one single method cannot provide a full description of script characteristics,so a combination of methods, both digital and manual, is desirable.

Logo and Seal serves the purpose of authenticating and referring to the source of a document. This strategy was also prevalent in the medieval period. Different algorithm exists for detection of logo and seal in document images. A close look into the present state-of-the-art methods reveals that those methods were focused toward detection of logo and seal in contemporary document images. However, such methods are likely to underperform while dealing with historical documents. This is due to the fact that historical documents are attributed with additional challenges like extra noise, bleed-through effect, blurred foreground elements and low contrast. The proposed method frames the problem of the logo and seals detection in an object detection framework. Using a deep-learning technique it counters earlier mentioned problems and evades the need for any pre-processing stage like layout analysis and/or binarization in the system pipeline. The experiments were conducted on historical images from 12th to the 16th century and the results obtained were very encouraging for detecting logo in historical document images. To the best of our knowledge, this is the first attempt on logo detection in historical document images using an object-detection based approach.

Ability to learn from a single instance is something unique to the human species and One-shot learning algorithms try to mimic this special capability. On the other hand, despite the fantastic performance of Deep Learning-based methods on various image classification problems, performance often depends having on a huge number of annotated training samples per class. This fact is certainly a hindrance in deploying deep neural network-based systems in many real-life applications like face recognition. Furthermore, an addition of a new class to the system will require the need to re-train the whole system from scratch. Nevertheless, the prowess of deep learned features could also not be ignored. This research aims to combine the best of deep learned features with a traditional One-Shot learning framework. Results obtained on 2 publicly available datasets are very encouraging achieving over 90% accuracy on 5-way One-Shot tasks, and 84% on 50-way One-Shot problems.

Realistic handwritten document generation garners a lot ofinterest from the document research community for its abilityto generate annotated data. In the current approach we haveused GAN-based stro ...

Binarization of digital documents is the task of classifying each pixel in an image of the document as belonging to the background (parchment/paper) or foreground (text/ink). Historical documents are often subjected to degradations, that make the task challenging. In the current work a deep neural network architecture is proposed that combines a fully convolutional network with an unrolled primal-dual network that can be trained end-to-end to achieve state of the art binarization on four out of seven datasets. Document binarization is formulated as an energy minimization problem. A fully convolutional neural network is trained for semantic segmentation of pixels that provides labeling cost associated with each pixel. This cost estimate is refined along the edges to compensate for any over or under estimation of the foreground class using a primal-dual approach. We provide necessary overview on proximal operator that facilitates theoretical underpinning required to train a primal-dual network using a gradient descent algorithm. Numerical instabilities encountered due to the recurrent nature of primal-dual approach are handled. We provide experimental results on document binarization competition dataset along with network changes and hyperparameter tuning required for stability and performance of the network. The network when pre-trained on synthetic dataset performs better as per the competition metrics.

We address the problem of segmenting and retrieving word images in collections of historical manuscripts given a text query. This is commonly referred to as "word spotting". To this end, we first propose an end-to-end trainable model based on deep neural networks that we dub Ctrl-F-Net. The model simultaneously generates region proposals and embeds them into a word embedding space, wherein a search is performed. We further introduce a simplified version called Ctrl-F-Mini. It is faster with similar performance, though it is limited to more easily segmented manuscripts. We evaluate both models on common benchmark datasets and surpass the previous state of the art. Finally, in collaboration with historians, we employ the Ctrl-F-Net to search within a large manuscript collection of over 100 thousand pages, written across two centuries. With only 11 training pages, we enable large scale data collection in manuscript-based historical research. This results in a speed up of data collection and the number of manuscripts processed by orders of magnitude. Given the time consuming manual work required to study old manuscripts in the humanities, quick and robust tools for word spotting has the potential to revolutionise domains like history, religion and language.

Most data mining applications on collections of historical documents require binarization of the digitized images as a pre-processing step. Historical documents are often subjected to degradations such as parchment aging, smudges and bleed through from the other side. The text is sometimes printed, but more often handwritten. Mathematical modeling of the appearance of the text, as well as the background and all kinds of degradations, is challenging. In the current work we try to tackle binarization as pixel classification problem. We first apply semantic segmentation, using fully convolutional neural networks. In order to improve the sharpness of the result, we then apply a graph cut algorithm. The labels from the semantic segmentation are used as source and sink estimates, with the probability map used for pruning the edges in the graph cut. The results obtained show significant improvement over the state of the art approach.

Most data mining applications on collections of historical documents require binarization of the digitized images as a pre-processing step. Historical documents are often subjected to degradations such as parchment aging, smudges and bleed through from the other side. The text is sometimes printed, but more often handwritten. Mathematical modeling of appearance of the text, background and all kinds of degradations, is challenging. In the current work we try to tackle binarization as pixel classification problem. We first apply semantic segmentation, using fully convolutional neural networks. In order to improve the sharpness of the result, we then apply a graph cut algorithm. The labels from the semantic segmentation are used as approximate estimates of the text and background, with the probability map of background used for pruning the edges in the graph cut. The results obtained show significant improvement over the state of the art approach.

In this paper, we approach the problem of segmentation-free query-by-string word spotting for handwritten documents. In other words, we use methods inspired from computer vision and machine learning to search for words in large collections of digitized manuscripts. In particular, we are interested in historical handwritten texts, which are often far more challenging than modern printed documents. This task is important, as it provides people with a way to quickly find what they are looking for in large collections that are tedious and difficult to read manually. To this end, we introduce an end-to-end trainable model based on deep neural networks that we call Ctrl-F-Net. Given a full manuscript page, the model simultaneously generates region proposals, and embeds these into a distributed word embedding space, where searches are performed. We evaluate the model on common benchmarks for handwritten word spotting, outperforming the previous state-of-the-art segmentation-free approaches by a large margin, and in some cases even segmentation-based approaches. One interesting real-life application of our approach is to help historians to find and count specific words in court records that are related to women's sustenance activities and division of labor. We provide promising preliminary experiments that validate our method on this task.

Visual understanding is a key aspect when studying electron tomography data-sets, aside quantitative assessments such as registration of high-resolution structures. We here present the free software tool ProViz (Protein Visualization) for visualisation and template matching in electron tomograms of biological samples. The ProViz software contains methods and tools which we have developed, adapted and computationally optimised for easy and intuitive visualisation and analysis of electron tomograms with low signal-to-noise ratio. ProViz complements existing software in the application field and serves as an easy and convenient tool for a first assessment and screening of the tomograms. It provides enhancements in three areas: (1) improved visualisation that makes connections as well as intensity differences between and within objects or structures easier to see and interpret, (2) interactive transfer function editing with direct visual result feedback using both piecewise linear functions and Gaussian function elements, (3) computationally optimised template matching and tools to visually assess and interactively explore the correlation results. The visualisation capabilities and features of ProViz are demonstrated on various biological volume data-sets: bacterial filament structures in vitro, a desmosome and the transmembrane cadherin connections therein in situ, and liposomes filled with doxorubicin in solution. The explorative template matching is demonstrated on a synthetic IgG data-set.

Deep learning has thus far not been used for dating of pre-modern handwritten documents. In this paper, we propose ways of using deep convolutional neural networks (CNNs) to estimate production dates for such manuscripts. In our approach, a CNN can either be used directly for estimating the production date or as a feature learning framework for other regression techniques. We explore the feature learning approach using Gaussian Processes regression and Support Vector Regression. The evaluation is performed on a unique large dataset of over 10000 medieval charters from the Swedish collection Svenskt Diplomatariums huvudkartotek (SDHK). We show that deep learning is applicable to the task of dating documents and that the performance is on average comparable to that of a human expert.

Finding the production date of a pre-modern manuscript is commonly a long process in historical research, requiring days of work from highly specialised experts. In this paper, we present an automatic dating method based on modelling both the language and the image data. By creating a statistical model over the changes in the pen strokes and short character sequences in the transcribed text, a combination of multiple estimators give a distribution over the time line for each manuscript. We have evaluated our estimation scheme on the medieval charter collection "Svenskt Diplomatariums huvudkartotek" (SDHK), including more than 5300 transcribed charters from the period 1135 - 1509. Our system is capable of achieving a median absolute error of 12 years, where the only human input is a transcription of the charter text. Since reading and transcribing the text is a skill that many researchers and students have, compared to the more specialized skill of dating medieval manuscripts based on palaeographical expertise, we find our novel approach suitable for helping individual researchers to date collections of manuscript pages. For larger collections, transcriptions could also be collected using crowd sourcing.

In the last few years, deep convolutional neural networks have become ubiquitous in computer vision, achieving state-of-the-art results on problems like object detection, semantic segmentation, and image captioning. However, they have not yet been widely investigated in the document analysis community. In this paper, we present a word spotting system based on convolutional neural networks. We train a network to extract a powerful image representation, which we then embed into a word embedding space. This allows us to perform word spotting using both query-by-string and query-by-example in a variety of word embedding spaces, both learned and handcrafted, for verbatim as well as semantic word spotting. Our novel approach is versatile and the evaluation shows that it outperforms the previous state-of-the-art for word spotting on standard datasets.

Hidden Markov Models constitute an established approach often employed for offline handwritten character recognition in digitized documents. The current work aims at evaluating a number of procedures frequently used to define features in the character recognition literature, within a common Hidden Markov Model framework. By separating model and feature structure, this should give a more clear indication of the relative advantage of different families of visual features used for character classification. The effects of model topologies and data normalization are also studied over two different handwritten datasets. The Hidden Markov Model framework is then used to generate images of handwritten characters, to give an accessible visual illustration of the power of different features.

In this paper a dynamic image representation is proposed by combining advantages of both raster and vector graphics in the triangulation. In order for a dynamic mesh to remain a triangulation, a method which maintains integrity of representation is devised. Together with techniques for synthesizing paint, the end result is a configurable scheme demonstrating potential as a viable alternative for digital painting and imaging in general.

In this paper, we introduce image-based word clouds as a novel tool for a quick and aesthetic overviews of common words in collections of digitized text manuscripts. While OCR can be used to enable summaries and search functionality to printed modern text, historical and handwritten documents remains a challenge. By segmenting and counting word images, without applying manual transcription or OCR, we have developed a method that can produce word or tag clouds from document collections. Our new tool is not limited to any specific kind of text. We make further contributions in ways of stop-word removal, class based feature weighting and visualization. An evaluation of the proposed tool includes comparisons with ground truth word clouds on handwritten marriage licenses from the 17th century and the George Washington database of handwritten letters, from the 18th century. Our experiments show that image-based word clouds capture the same information, albeit approximately, as the regular word clouds based on text data.

The segmentation of individual words is a crucial step in several data mining methods for historical handwritten documents. Examples of applications include visual searching for query words (word spotting) and character-by-character text recognition. In this paper, we present a novel method for word segmentation that is adapted from recent advances in computer vision, deep learning and generic object detection. Our method has unique capabilities and it has found practical use in our current research project. It can easily be trained for different kinds of historical documents, uses full gray scale information, does not require binarization as pre-processing or prior segmentation of individual text lines. We evaluate its performance using established error metrics, previously used in competitions for word segmentation, and demonstrate its usefulness for a 15th century handwritten document.