This paper presents two deep learning models using a multi-perspective convolutional neural network (CNN) for classifying objects in the context of intelligent transportation systems (ITS). The proposed model categorizes objects accurately, enabling them to make well-informed decisions in multi-object (such as Persons, Trucks, Motorbikes, Cars, and Cyclists.) detection in complex scenarios for automotive applications. The custom backbone model is designed based on experimentation with the VGG backbone network based on the VGG backbone network, incorporating a multilayer prediction head and custom feature extraction blocks for classifying multiple objects in complex scenes. The model is to extract abstract features and features at multiple scales with a custom-designed feature extraction backbone with multiple blocks. The proposed models are lightweight and require fewer computational resources for high classification performance. The automotive publicly available dataset with 19800 images and labels has been used. Results show that when we experimented with the VGG backbone CNN model, the classification accuracy of 99.64% is achieved, and on the other hand, the classification accuracy of custom backbone CNN is 99.46%. The performance of the proposed custom model is also compared to those of pre-trained benchmark models. The experimental findings presented in this paper show that the proposed models achieve higher accuracy than the pre-trained models.
Convolutional Neural Networks have become an important tool for various Computer Vision tasks. Yet, increasing complexity of such architectures drives computational costs. To this end, we propose two measures to achieve similar classification results as state-of-the-art architectures while at the same time reducing model complexity significantly. Firstly, we describe a novel type of non-linear parameter-efficient morphological layers inspired by concepts that are well-known and widely used with convolutions. Secondly, we present a set of simple network architectures, organized as optimization framework, which is enhanced by neural architecture search and hyperparameter optimization. In experiments with hyperspectral remote sensing data, we demonstrate that the identified optimal morphological architecture produces results not only comparable with other architectures from the optimization framework, but also comparable or better than selected state-of-the-art neural network architectures for image classification. Depending on the performed task, the proposed optimized architecture requires up to 25 times fewer parameters than actual state-of-the-art networks.
We present FARA, a novel approach for fast approximation of RFD-like descriptors in the context of document retrieval systems. RFD-like descriptors are widely used for document representation, but their computation is expensive, especially for large document collections. Our method is a CPU-friendly gradient maps computation approximation with sequential memory access and integer-only calculations. There are three types of operations that we use: addition, subtraction, and absolute values. It allows us to effectively use SIMD extensions, resulting in an additional increase in the running speed. Experimental results demonstrate that FARA achieves the same accuracy as RFDoc descriptors and significantly reduces the computational overhead. The proposed approach achieves a twofold speed improvement of gradient maps computation and 25% acceleration of overall descriptor computing time compared to the most efficient RFDoc implementation.
The human olfactory bulb (OB), an important part of the brain responsible for the sense of smell, is a complex structure composed of multiple layers and cell types. Studying the OB morphological structure is essential for understanding the decline in olfactory function related to aging, neurodegenerative disorders, and other pathologies. Traditional microscopy methods in which slices are stained with solutions to contrast individual elements of the morphological structure are destructive. Non-destructive high-resolution technique is the X-ray phase-contrast tomography. However, manual segmentation of the reconstructed images are time-consuming due to large amount of data and prone to errors. U-Net-based model to optimize the segmentation of OB morphological structures, focusing specifically on glomeruli, in tomographic images of the human OB is proposed. The strategy to address overfitting and enhance the model's accuracy is described. This method addresses the challenges posed by complex limited data containing abundant details, similar grayscale levels between soft tissues, and blurry image details. Additionally, it successfully overcomes the limitations of a small dataset containing images with extremely dense point clouds, preventing the models from overfitting.
Nowadays, mobile robots solve various tasks realted to navigation in an unknown or changing environment. In such conditions, mapping the environment is crucial for mobile robot navigation. Conventionally, maps are built as 2D or 3D dense metric structures which require much memory for storage and much computational time for path planning, especially in large environments. Representing a map as a topological structure (i.e. a graph of locations) allows fast path planning with low memory consumption. In this paper, we present a method of real-time topological map building and updating from odometry measurements and local point clouds. The proposed method guarantees the topological graph connectivity and adds shortcuts to the graph to optimize paths. We tested our method in a simulated environment and measured efficiency of path planning in the obtained graphs. In all the tests, the SPL value exceeded 81%, with 100% success rate. The source code of our method is available at https://github.com/KirillMouraviev/simple_toposlam_model.
It is critical to make full use of the information of the backbone to improve the performance of object tracking. A common way to mine useful information is to add attention to features. However, most trackers only use single attention to mine the features and fail to utilize the effective information in the backbone. To leverage the useful information of features in multiple ways, this paper proposes an integrated channel attention mechanism based on all kinds of commonly used channel attention methods. First, we used ResNet50 as the backbone, and then we used four attention methods to process the fourth-stage features that were extracted from the backbone to obtain attention factors. Then, through adaptive weighting, we added the four attention methods to the original features. It adaptively adjusts the importance of each channel attention, suppresses redundant information, and better captures key features of the tracked object in different channels. The effectiveness of our approach is validated on five tracking benchmarks.
The ability of autonomous vehicles (AVs) to detect three-dimensional objects is crucial for motion planning, object tracking and safe driving. This task is especially challenging for systems using only monocular cameras, for which depth estimation presents special difficulties. In this paper, we discuss the subsystem of 3D object detection in bird's-eye-view (BEV) for a single camera in an AV system. The subsystem consists of two parts. First, it estimates the contour of the object's projection polygon in BEV based on 2D detection and drivable area segmentation (a planar ground model is used). Second, it simplifies the object's projection by fitting the obtained polygon to a rotated bounding box. For this part we propose a new L-shape model-based fitting algorithm. It assumes that the vertices of the input polygon belong to two adjacent sides of the fitted bounding box. We compared this algorithm with a naive approach which minimizes the bounding box's area and with adaptations of algorithms from a paper solving a similar problem with LiDAR point clouds. The L-shape algorithm outperformed the alternatives.
Spectral reconstruction (recovering spectra from RGB measurements) is a vital problem of computational photography. As a matter of curiosity, modern mobile devices open a new opportunity to improve the quality of spectral reconstruction by utilizing images from several cameras at once. This leads to the idea of creating a mobile hyperspectral camera for the general public. In this paper we investigate the achievable accuracy when using several identical cameras simultaneously in combination with different spectral filters. To find optimal filters, two algorithms are proposed: one learns spectral transmittance functions simultaneously with spectral reconstruction, the other learns only spectral transmittances by information loss minimization. As a result of numerical experiments, 4 cameras and 4 filters allow us to perform spectral reconstruction two times accurately than from a single RGB image.
Object detection is a critical task in computer vision with applications across various domains, ranging from autonomous driving to surveillance systems. Despite extensive research on improving the performance of object detection systems, identifying all objects in different places remains a challenge. The traditional object detection approaches focus primarily on extracting and analyzing visual features without considering the contextual information about the places of objects. However, entities in many real-world scenarios closely relate to their surrounding environment, providing crucial contextual cues for accurate detection. This study investigates the importance and impact of places of images (indoor and outdoor) on object detection accuracy. To this purpose, we propose an approach that first categorizes images into two distinct categories: indoor and outdoor. We then train and evaluate three object detection models (indoor, outdoor, and general models) based on YOLOv5 and 19 classes of the PASCAL VOC dataset and 79 classes of COCO dataset that consider places. The experimental evaluations show that the specialized indoor and outdoor models have higher mAP (mean Average Precision) to detect objects in specific environments compared to the general model that detects objects found both indoors and outdoors. Indeed, the network can detect objects more accurately in similar places with common characteristics due to semantic relationships between objects and their surroundings, and the network's misdetection is diminished. All the results were analyzed statistically with t-tests.
In modern agriculture, livestock monitoring plays a vital role in ensuring animal health, welfare, and production efficiency. Leveraging computer vision and deep learning, this paper presents an innovative framework aimed at enhancing livestock monitoring. Specifically, we address two crucial challenges: denoising and segmentation of cattle in livestock images. The denoising task is fundamental in preprocessing noisy images affected by adverse environmental conditions and equipment limitations. To tackle this, we introduce an encoder-decoder model that effectively denoises cattle images while preserving critical anatomical details. Our framework incorporates a segmentation module inspired by the U-Net architecture. Notably, both denoising and segmentation tasks share a common encoder, optimizing computational efficiency. The segmentation model employs hybrid loss functions and leverages the Grad-CAM technique to provide interpretable insights into the decision-making process. Our approach stands as one of the pioneering joint solutions for cattle denoising and segmentation, particularly focusing on top-view cattle images.
We introduce a novel approach for image edge detection based on calculating pseudo-Boolean polynomials on image patches whose resulting polynomial degrees determine whether a patch lies over an edge or a blob. In this paper we show that patches covering edge regions within the image result in pseudo-Boolean polynomials of higher degrees compared to patches that cover blob regions. The proposed approach is based on reduction of polynomial degree and equivalence properties of penalty-based pseudo-Boolean polynomials.
Selecting representative data is a key factor in improving the performance of machine learning algorithms. In this paper we focus on out-of-distribution (OoD) methods evaluation, which can be integrated into ML project lifecycle in a nonintrusive way, without changing a model architecture. Considered methods are applicable to image classification datasets analysis. In addition to commonly used AUROC metric, we evaluate the number of out-of-distribution samples misclassified with high confidence. Case studies were conducted on benchmark and production datasets. As a result, we provide practical guidance for data evaluation and recommendations on which method to use to detect different types of OoD images.
The amount of image forgery strongly increases recently. There are different ways to fake an image, one of the common ones is a copy-move manipulation. There are numerous methods for detecting copy-move manipulations on natural images. However, they are difficult to adapt for document images due to their features. This work proposes an algorithm for detecting and localizing copy-move manipulations on digital images of documents. The main idea is to use JPEG artifacts in order to find the target region area and then localize the source and target regions precisely. For the efficient application of the proposed method, firstly, the original image must have been subjected to JPEG compression, and secondly, after the manipulation the image must have been saved in a lossless format. The experiments were carried out on an open set of document images CMID; in the detection task, the recall was 0.992, the specificity was 1.0; in the localization task, the recall was 0.923, the false discovery rate was 0.021, which means that the proposed algorithm successfully detects more than 99% of copy-move manipulations, similar to manipulations in the CMID and does not give false positives.
There are various techniques for decreasing the computational complexity of neural networks, and a number of them use neuron approximations. A bipolar morphological neuron is an approximation of a classical neuron that can be used on FPGAs and ASICs to enhance computational efficiency. It uses 4 distinct computational pathways utilizing addition and maximum functions, in contrast to the traditional neuron which employs multiplication and addition. In this paper, we introduce bipolar morphological YOLO network for object detection task. To train the network, we employ an iterative approach that combines knowledge distillation for backbone and fine-tuning of the network's head. Our experiments, which were conducted using the COCO dataset, yield results that are on par with classical networks. Specifically, the average recall for large images is 0.393 for the BM network and 0.371 for the classical network. Additionally, the average precision values are 0.088 for the BM network and 0.097 for the classical network. These outcomes establish a baseline for object detection using bipolar morphological networks.
One-class incremental learning is a special case of class-incremental learning, where only a single novel class is incrementally added to an existing classifier instead of multiple classes. This case is relevant in industrial defect detection scenarios, where novel defects usually appear during operation. Existing rolled-out classifiers must be updated incrementally in this scenario with only a few novel examples. In addition, it is often required that the base classifier must not be altered due to approval and warranty restrictions. While simple finetuning often gives the best performance across old and new classes, it comes with the drawback of potentially losing performance on the base classes (catastrophic forgetting [1]). Simple prototype approaches [2] work without changing existing weights and perform very well when the classes are well separated but fail dramatically when not. In theory, null-space training (NSCL) [3] should retain the basis classifier entirely, as parameter updates are restricted to the null space of the network with respect to existing classes. However, as we show, this technique promotes overfitting in the case of one-class incremental learning. In our experiments, we found that unconstrained weight growth in null space is the underlying issue, leading us to propose a regularization term (R-NSCL) that penalizes the magnitude of amplification. The regularization term is added to the standard classification loss and stabilizes null-space training in the one-class scenario by counteracting overfitting. We test the method's capabilities on two industrial datasets, namely AITEX and MVTec, and compare the performance to state-of-the-art algorithms for class-incremental learning.
The right ventricular (RV) function deterioration strongly predicts clinical outcomes in numerous circumstances. To boost the clinical deployment of ensemble regression methods that quantify RV volumes using tabular data from the widely available two-dimensional echocardiography (2DE), we propose to complement the volume predictions with uncertainty scores. To this end, we employ an instance-based method which uses the learned tree structure to identify the nearest training samples to a target instance and then uses a number of distribution types to more flexibly model the output. The probabilistic and point-prediction performances of the proposed framework are evaluated on a relatively small-scale dataset, comprising 100 end-diastolic and end-systolic RV volumes. The reference values for point performance were obtained from MRI. The results demonstrate that our flexible approach yields improved probabilistic and point performances over other state-of-the-art methods. The appropriateness of the proposed framework is showcased by providing exemplar cases. The estimated uncertainty embodies both aleatoric and epistemic types. This work aligns with trustworthy artificial intelligence since it can be used to enhance the decision-making process and reduce risks. The feature importance scores of our framework can be exploited to reduce the number of required 2DE views which could enhance the proposed pipeline's clinical application.
Handwritten text recognition (HTR) is a challenging task that requires a large amount of diverse training data. One of the possible approaches to this problem is the adoption of CNNs. The key challenge is that the CNN requires geometrically labeled training data, which may increase the cost and time consumption of labeling. To overcome these limitations we propose the method, based on Generative Adversarial Network (GAN), which transfers handwriting styles to printed style images, preserving the Same geometrical Annotation as Input - SAIGAN. Taking printed style image as an input, it produces the handwritten image with the same text content located in the same positions. Our method operates on the character-level and can produce sequences of an arbitrary length and any content. Once trained, it is also possible to generate new handwriting styles by simply manipulating latent vectors. Proposed character style supervision allowed our model to surpass the basis method.
In this paper, we consider the problem of clothes compatibility for total look recommendation systems by means of deep neural networks. This task has become very popular in recent years, primarily due to the growth of online retail sales of clothing. Unlike the existing solutions, we developed a comprehensive model of clothes compatibility evaluation based on color characteristics as well as on the characteristics of the style. As a rule, neural networks are robust to the color characteristics of an image, but color is an extremely important component in the task of a total look evaluation, so such additional branch with color characteristics is well justified. The proposed model uses both: color embedding obtained from color clustering and histograms, and style embedding as an output tensor of ResNet-50 encoder. The paper shows that color embeddings significantly improve the quality of the total look evaluation. The model was trained on Polyvore dataset, which was pre-processed and cleaned from the items not related to the topic of total look compatibility.
In the paper, we present a quantization method for bipolar morphological neural networks. Bipolar morphological neural networks use only addition, subtraction, and maximum operations inside the neuron and exponent and logarithm as activation functions of the layers. These operations allow fast and compact gate implementation for FPGA and ASIC, which makes these networks a promising solution for embedded devices. Quantization allows us to reach an additional increase in computational efficiency and reduce the complexity of hardware implementation by using integer values of low bitwidth for computations. We propose an 8-bit quantization scheme based on integer maximum, addition, and lookup tables for non-linear functions and experimentally demonstrate that basic models for image classification can be quantized without noticeable accuracy loss. More advanced models still provide high recognition accuracy but would benefit from further fine-tuning.
We propose a novel active learning framework for image classification - sMoBYAL. Our contribution is modifying MoBY - one of the highly effective self-supervised learning algorithms to utilize both labeled and unlabeled data for the active learning pipeline. Finally, we thoroughly evaluate and analyze the robustness and performance of our pipeline in image classification tasks. Our approach attains comparative outcomes, surpassing recent AL methods in terms of results. Our code available at: https://github.com/thanhdh-3030/sMoBYAL
