This companion paper provides a detailed account of the EMPATH framework [8], which integrates Ensemble Learning, MediaPipe Holistic for gesture tracking, and an Attention-based Transformer model, along with practical insights on designing an effective recognition model. It focuses on the architecture, workflow and offers an in-depth explanation of the interpolation method for handling missing hand keypoints. Additionally, the paper highlights ablation studies on augmentation techniques and key limitations encountered during development and implementation, providing valuable insights for enhancing the reproducibility and performance of word-level sign language recognition using EMPATH.
As a companion to the ICPR 2024 accepted paper "SalFoM: Dynamic Saliency Prediction with Video Foundation Models", this work investigates how various model parameters and components impact its performance. Since SalFoM represents the first effort of its kind in this field, the additional experiments presented here are designed to provide insights into the application of video foundation models for dynamic saliency prediction. This is achieved by exploring different aspects of the model's architecture and the use of large video models. Additionally, this work analyzes the impact of various strategies for defining training objectives on the model's learning capabilities and overall performance. The code is available at https://github.com/mr17m/SalFoM-Video-Saliency-Prediction.
This paper provides a detailed description of how to install, set up, and use "BarBeR" (Barcode Benchmark Repository) to reproduce the results presented in the ICPR 2024 paper "BarBeR: A Barcode Benchmarking Repository". The paper details the tests available in the repository and how the configuration parameters affect and influence experimental results.
Automated Vickers hardness measurement methods must be robust and versatile to cope with materials with rough and noisy patterned surfaces. The behaviour of several state-of-the-art algorithms for automated Vickers hardness measurement is evaluated and compared for their respective robustness and accuracy. The evaluation is based on real world industrial data, collected in a Vickers indentation dataset. Addressing the current lack of publicly available (annotated) Vickers indentation data, and in order to provide a benchmark for further investigations and experiments, we release the databases used in this work, together with expert groundtruth covering diagonal measurement and segmentation masks, and make them available on-line for public open access.
This article is a companion paper to our publication for ICPR 2024, "FNOReg: Resolution-Robust Medical Image Registration Method Based on Fourier Neural Operator" [7], in which we proposed an algorithm for medical image registration that offers robustness to input data resolution. This companion paper presents the method's implementation, the methodology for hyperparameter selection, and other details that ensure the reproducibility of our research.
Scratch assay evaluation is a widely used and standardized method for investigating collective cell migration and wound healing processes. Modern laboratory technology offers the possibility of highresolution digital image acquisition leading to the use of computer vision methods for the analysis of scratch assays.Benchmarking is necessary to measure the performance of newly developed analytical tools. Publicly available data sets and software tools are needed to ensure objective and reproducible testing. This work presents a new dataset consisting of highresolution raw images obtained with an industrial plate reader together with ground truth data manually annotated by experts to overcome the sparse availability of annotated scratch assay images. The advantage of this dataset over other publicly available datasets is its much higher resolution (up to 10 times). It also covers the entire cell layer in a well, allowing the entire wound area to be examined rather than just the central region. In addition to the dataset, a software tool will be published containing two algorithms developed to measure wound healing using the presented data. Their performance will be compared with other publicly available tools.
This is the research reproducibility paper for the ICPR 2024 paper "Learning Neural Networks for Multi-label Medical Image Retrieval Using Hamming Distance Fabricated with Jaccard Similarity Coefficient". This work provides an overview of the problem, highlighting the motivation behind multi-label image retrieval in the medical domain. It delves into the algorithmic framework, offering a detailed explanation of the proposed solution, including illustrative examples. Additionally, we outline the implementation guidelines for replicating the approach. A thorough experimental evaluation emphasizes the employed loss functions, parameter analysis, and an ablation study. The limitations of this method are discussed at the end.
Reproducibility is a crucial aspect of neural network training, but it remains challenging due to the stochastic nature of the optimization during the training process. This paper examines the effects of two weight optimization techniques: weight selection optimization and traditional weight update optimization, on the run-to-run variability of neural network performance. Our extensive experiments across various convolutional layer sizes reveal that weight selection optimization reduces variability by approximately 25% to 40%, depending on the model, compared to traditional training methods. However, while variability is reduced, model accuracy remains unchanged, staying on par with conventional training. These findings provide valuable insights into enhancing the stability and reproducibility of neural network models without compromising accuracy.
This companion paper focuses on the reproducibility of our previously accepted work on the proposed MeDiANet architecture, a lightweight computationally efficient architecture designed for medical image classification. We outline key practices, including consistent random seed initialization and mixed-precision training with TensorFlow. By detailing these essential steps, we provide a clear framework to enable other researchers to replicate the findings, ensuring transparency and reliability in deep learning experiments. The code is available at: https:// github.com/dipayandewan94/MeDiANet.
Reproducibility in terms of implementation and performance is a crucial aspect of healthcare applications as they can have high impact consequences on patient welfare and safety. In this paper, we focus on the consistency and reproduction of results for graph neural networks (GNN) based facial palsy and paresis evaluation. Comparative studies between our proposed GNN-based model and state-of-the-art (SOTA) convolutional neural network-based models suggest that the GNN model is sensitive to pose variability within the dataset while the CNN-based models are consistent across the board. With these findings, we propose a sufficiently regularised dataset with pose variability for obtaining consistent and better results. We provide further analysis of the classification behaviour of our model, the results of which suggest potential label ambiguity within the dataset employed. Future improvements regarding the model's performance and consistency are recommended based on the reproducibility analyses.
This is the companion paper to the ICPR 2024 Paper "GolfPose: From Regular Posture to Golf Swing Posture". It details our GolfSwing dataset and GolfPose model. For the GolfSwing dataset, we introduce the equipment used, data recording steps, and post-processing methods. Regarding the GolfPose models, we discuss parameters for fine-tuning both object detectors and 2D/3D pose models, along with integration methods for inference. Finally, we present metrics to evaluate the performance of 2D and 3D pose models and compare the 2D pose model with a pre-trained model.
Smart textiles with embedded capacitive touch sensors (CTS) hold great promise for intuitive gesture-based human-computer interaction. However, recognizing complex gestures in real-time on resource-constrained wearable devices remains a challenge. This paper presents a novel approach using a minimalist neural network architecture tailored for efficient gesture recognition on knitted CTS. We emphasize reproducibility throughout our work by providing detailed algorithmic implementation, the influence of key parameters on result quality, and the integration of our source code into other frameworks. Our method demonstrates the ability to accurately classify a variety of single- and multi-touch gestures, including taps, swipes, and pinches, with accuracy rates exceeding 90 https://github.com/dsbuddy/knitted-capacitive-touch-sensor-gesture-recognition .
With the rapid advances in deep learning and smart manufacturing in Industry 4.0, there is an imperative for high-throughput, high-performance, and fully integrated visual inspection systems. Most anomaly detection approaches using defect detection datasets, such as MVTec AD, employ one-class models that require fitting separate models for each class. On the contrary, unified models eliminate the need for fitting separate models for each class and significantly reduce cost and memory requirements. Thus, in this work, we experiment with considering a unified multi-class setup. Our experimental study shows that multi-class models perform at par with one-class models for the standard MVTec AD dataset. Hence, this indicates that there may not be a need to learn separate object/class-wise models when the object classes are significantly different from each other, as is the case of the dataset considered. Furthermore, we have deployed three different unified lightweight architectures on the CPU and an edge device (NVIDIA Jetson Xavier NX). We analyze the quantized multi-class anomaly detection models in terms of latency and memory requirements for deployment on the edge device while comparing quantization-aware training (QAT) and post-training quantization (PTQ) for performance at different precision widths. In addition, we explored two different methods of calibration required in post-training scenarios and show that one of them performs notably better, highlighting its importance for unsupervised tasks. Due to quantization, the performance drop in PTQ is further compensated by QAT, which yields at par performance with the original 32-bit Floating point in two of the models considered.
This paper presents a detailed and reproducible description of the algorithms and experiments published in our ICPR paper ( A novel pattern-based edit distance for automatic log parsing ). It discusses the implementation, our methodology, our experimental setup, the considered performance metrics and the influence of the main parameters of the compared algorithms.
Accurate location of access roads is important for forest management, in particular in mountain areas. In this paper, we are interested in their detection from LiDAR data using deep learning approaches. For this, we use images computed from an interpolated surface, called digital terrain model (DTM), of the 3D point cloud. In order to train and validate the neural network models, two ground truth datasets associated to DTM images are considered: (1) manual digitization of the road centerlines and (2) automatic extraction followed by supervised completion using two softwares based on discrete geometry tools. The trained network models are then evaluated over a test dataset using standard measures such as precision, recall, F-measure and prediction time.
Reproducible research in pattern recognition can be viewed from a number of angles, including code execution, platforms that promote reproducibility, code sharing, or the release of libraries providing access to relevant algorithms in the corresponding disciplines. In this work, after recalling the motivation and classic definitions of reproducible research, we propose an updated overview of the main platforms that might be used for reproducible research. We then review the different libraries that are commonly used by the pattern recognition, computer vision, imaging and geometry processing communities, and we share our experience of developing a research library. In the third part, new advanced editorial investments will be presented, such as the IPOL journal or other IPOL-inspired new initiatives like OVD-SaaS.
Business documents are used every day by all kinds and sizes of companies and administrations, even if most of these entities have several information systems where the documents are digitilized in different formats (json, xml, database tables, ...), there still an important number of business documents that require manual processing which costs a lot and is very time consuming. Extracting key-value information from business documents is a challenging problem due to the variety of document types and templates, in this work we will deal with the problem as a graph node classification problem using a multi “graph transformer" layers, we propose a graph construction method that focuses on the most relevant neighbours of every node while reducing the size of the graph and we use a document transformer embedding combined with some spatial and textual feature to give a better representation to each node. Our contribution in this work was to conceive a graph neural network (GNN) achieving the highest results comparing to the rest of GNN models dealing with the same problem to our knowledge, the model is small (53,6K parameters) comparing to the recent models using transformers architectures (hundreds of millions of parameters) which is very suitable for applications when storage constraints are present, it also has a limited impact on the environment and represent an alternative to build greener AI systems. We experiment our model on the SROIE ICDAR receipts dataset where we got an important F1 score compared to other graph neural network (GNN) based models.
Following the fourth edition of the workshop on Reproducible Research in Pattern Recognition (RRPR) at the International Conference on Pattern Recognition (ICPR), this paper reports the main discussions that were held during and after the workshop. In particular, the integration of reproducible research inside an international conference was the first main axis of reflection. Further discussions addressed the ways of initiating or imposing reproducible research, as well as the problem of performance comparisons of published research papers that emerges due to the fact that the reported results are often based on different implementations and datasets.
Max-tree (or component-tree) is a hierarchical representation which associates to a scalar image a descriptive data structure induced by the inclusion relation between the binary components obtained at successive level-sets. Various attributes related to these binary components can be computed and stored into the tree. Max-trees have been involved in many applications, enabling to perform attribute filtering in an efficient algorithmic way. Since the resulting images do not contain any new contour, these kind of filters are called connected operators . In this paper, we propose to rely on max-trees and attribute filters to enrich the input of a convolutional neural network (CNN) to improve a task of segmentation. More precisely, two approaches are considered: a first approach in which images are preprocessed using attribute filters and a second approach in which maps of attributes relying on max-trees are computed. Based on these two different approaches, the resulting maps are used as additional input in a standard CNN in a context of semantic segmentation. We propose to compare different attributes and nodes selection strategies and to experiment their usage on a practical problem: the segmentation of the mitochondria and endoplasmic-reticulum in Focused Ion Beam milling combined with Scanning Electron Microscopy (FIB-SEM) images. We provide original images, annotations, source code and a documentation to reproduce the experimentation results.
This companion paper focuses on challenging aspects of reproducibility that emerge in anomaly detection with Deep Neural Networks. We provide motivating examples based on our work and present mitigation strategies. Furthermore, we document a trade-off between the complexity of experiments and the strength of the empirical evidence obtained through them, both of which impact different types of reproducibility. Ultimately, we argue that the reproducibility of inferences should be prioritized over the reproducibility of exact numerical results.
