Handwritten Text Recognition (htr) is more interesting and challenging than printed text due to uneven variations in the handwriting style of the writers, content, and time. htr becomes more challenging for the Indic languages because of (i) multiple characters combined to form conjuncts which increase the number of characters of respective languages, and (ii) near to 100 unique basic Unicode characters in each Indic script. Recently, many recognition methods based on the encoder-decoder framework have been proposed to handle such problems. They still face many challenges, such as image blur and incomplete characters due to varying writing styles and ink density. We argue that most encoder-decoder methods are based on local visual features without explicit global semantic information. In this work, we enhance the performance of Indic handwritten text recognizers using global semantic information. We use a semantic module in an encoder-decoder framework for extracting global semantic information to recognize the Indic handwritten texts. The semantic information is used in both the encoder for supervision and the decoder for initialization. The semantic information is predicted from the word embedding of a pre-trained language model. Extensive experiments demonstrate that the proposed framework achieves state-of-the-art results on handwritten texts of ten Indic languages.
Handwritten text recognition is still an unsolved problem in the field of machine learning. Nevertheless, this technology has improved considerably in the last decade in part thanks to advancements in recurrent neural networks. Unfortunately, due to their sequential nature, recurrent models cannot be effectively parallelised during training. Meanwhile, in natural language processing research, the transformer has recently become the dominant architecture; replacing the recurrent networks that were once popular. These new models are far more efficient to train than their predecessors because their primary building block, the self-attention network, can process sequences entirely non-recurrently. This work demonstrates that self-attention networks can replace the recurrent networks of state-of-the-art handwriting recognition models and achieve competitive error rates, while reducing the time required to train and the number of parameters significantly.
The variety of handwriting styles and the scarcity of training data often result in poor performance of character recognizer. Rather than tedious data collection and annotation, researchers prefer to use low-cost data augmentation to improve the robustness of the recognizer. However, most existing data augmentation methods treat handwritten text as ordinary images and generate new samples through holistic transformation, which greatly limits the diversity of generated samples. To solve the problem, this paper proposes a script-level handwritten text augmentation method, where each component is treated as a Bézier curve. Specifically, we first segment the character into components based on skeleton detection. Then, we move the control points of each component according to the prior knowledge of languages. Finally, we transform the component by Bézier curves and assemble them into new samples. Our method is simple, controllable, and friendly to few-shot handwritten text. Experiments on four datasets in different languages show that the proposed script-level augmentation method performs better than the holistic augmentation methods. Apart from it, we also modify Affine transformation, a commonly used augmentation method, from a holistic to script-level way. Experimental results demonstrate that script-level affine can achieve better performance than holistic affine in the character recognition task. Our code is available at https://github.com/IMU-MachineLearningSXD/script-level_aug_ICFHR2022 .
Handwritten Text Recognition (HTR) relies on deep learning to achieve high performances. Its success is substantially driven by large annotated training datasets resulting in powerful recognition models. Performances suffer considerably when applied to document collections with a distinctive style that is not well represented by training data. Applying a recognition model to a new data collection poses a tremendous annotation effort, which is often out of scope, for example considering historic collections. To overcome this limitation, we propose a training scheme that combines multiple data sources. Synthetically generated samples are used to train an initial model. Self-training offers the possibility to exploit unlabeled samples. We further investigate the question of how a small number of manually annotated samples can be integrated to achieve maximal performance with limited annotation effort. Therefore, we add labeled samples at different stages of self-training and propose two criteria, namely confidence and diversity, for the selection of samples to annotate. In our experiments, we show that the proposed training scheme is able to considerably close the gap to fully-supervised training on the designated training set with less than ten percent of the labeling demand.
Digitization of ancient palm leaf manuscripts is gaining momentum due to the limited datasets and complex features of text images of palm leaf manuscripts. Thus far, the previous studies did not deeply analyze the application of the trending techniques on the palm leaf manuscripts, considering how deep learning approaches require large datasets, while some isolated glyphs contain more than one character with complex grammatical components. Therefore, this paper explores the possibilities and practical methods for improving isolated glyph classification. In particular, we focus on both the front-end and the back-end processes involved in the image classification task. For the front-end analysis, we present multi-task preprocessing techniques, including data augmentation techniques, new datasets extraction, and image enhancement techniques to increase the quality and quantity of datasets. For the back-end side, we aim to study the visual backbones of deep learning techniques, especially CNNs (including VGG, ResNet, and EfficientNet) and attention-based models (including ViT, DeiT, and CvT). Furthermore, the analysis and evaluation examined how data augmentation techniques and preprocessing interact with the amount of data used in training. Evidently, we experimented on three palm leaf manuscripts, including Balinese, Sundanese, and Khmer scripts from the ICFHR contest 2018, SluekRith, AMDI LontarSet, and Sunda datasets. Regarding the quality of research, the experiment delivers an effective way of training palm leaf datasets for the document analysis community.
