Deep learning models can perform well when evaluated on images from the same distribution as the training set. However, applying small perturbations in the forms of noise, artifacts, occlusions, blurring, etc. to a model's input image and feeding the model with out-of-distribution (OOD) data can significantly drop the model's accuracy, making it not applicable to real-world scenarios. Data augmentation is one of the well-practiced methods to improve model robustness against OOD data; however, examining which augmentation type to choose and how it affects the OOD robustness remains understudied. There is a growing belief that augmenting datasets using data augmentations that improve a model's bias to shape-based features rather than texture-based features results in increased OOD robustness for Convolutional Neural Networks trained on the ImageNet-1K dataset. This is usually stated as “an increase in the model's shape bias results in an increase in its OOD robustness”. Based on this hypothesis, some works in the literature aim to find augmentations with higher effects on model shape bias and use those for data augmentation. By evaluating 39 types of data augmentations on a widely used OOD dataset, we demonstrate the impact of each data augmentation on the model's robustness to OOD data and further show that the mentioned hypothesis is not true; an increase in shape bias does not necessarily result in higher OOD robustness. By analyzing the results, we also find some biases in the ImageNet-1K dataset that can easily be reduced using proper data augmentation. Our evaluation results further show that there is not necessarily a trade-off between in-domain accuracy and OOD robustness, and choosing the proper augmentations can help increase both in-domain accuracy and OOD robustness simultaneously.
Modern approaches to scene graph generation still struggle with their performance, with even state of the art approaches hovering under a 15% mean recall on certain evaluation modes. This poor performance is partially a result of networks heavily relying and fixating on non-visual data, such as class statistics, instead of the pixel-level signals present in the images. We demonstrate this by examining the 'visual-ness' of visual relationship detection approaches. We first describe and implement a new Naive Bayes-based statistical baseline for scene graph generation. Most notably, this basic classifier does not utilize the image pixels, but relies on the properties of the bounding boxes (class labels, topological configuration, … etc.) to predict the relationship labels. We demonstrate that our classical machine learning approach, one as simple as a categorical Naive Bayes classifier, can perform relationship detection in a manner that achieves relatively competitive performance to that of modern scene graph generators. This is an alarming finding regarding scene graph generation that implies that visual data in images may not be utilized in modern visual relationship detection past the point of object detection. We finally discuss how more visual modern approaches to scene graph generation appear to remedy some of these shortcomings.
Target tracking is a classic problem in computer vision, with numerous applications in robotics. However, tracking targets underwater presents additional complications due to the six degrees of freedom nature of the problem and the challenging visual environment. In this paper, we address the problem of robotic underwater tracking of scuba divers by partitioning it into two parts: vision and control. We propose a new approach that exploits a highly-maneuverable underwater robot to perform experiments in open water, coupling sensing and control for improved performance. To evaluate the temporal stability of different tracking paradigms, we introduce a new metric, frame-to-frame vari-ance, which is better suited to assess the smoothness of detections from the vision side. We implement PID controllers for control and a spiral search algorithm for target recovery in case of a tracking failure. Our approach only uses observations in the image plane, eliminating the need for robot localization or camera calibration. Using a tracking-by-detection paradigm that combines YOLOv7 for target detection, a tuned filtering technique for temporal stability, and a spiral search algorithm for target recovery, we demonstrate promising performance for long-term tracking. We evaluate our proposed paradigm on the VDD-C dataset and deploy it on an underwater robot for several experiments in open water. Our outcomes show consistency with the ones in the initial studies, and the spiral search algorithm demonstrates promising performance for recapturing a target after a tracking failure. Our approach delivers promising performance for robust underwater tracking, achieving successful open-water tracking scenarios in the presence of strong water currents.
This paper analyzes the robustness of recent 3D shape descriptors to SO(3) rotations, something that is fundamental to shape modeling. Specifically, we formulate the task of rotated 3D object instance detection. To do so, we consider a database of 3D indoor scenes, where objects occur in different orientations. We benchmark different methods for feature extraction and classification in the context of this task. We systematically contrast different choices in a variety of experimental settings investigating the impact on the performance of different rotation distributions, different degrees of partial observations on the object, and the different levels of difficulty of negative pairs. Our study, on a synthetic dataset of 3D scenes where objects instances occur in different orientations, reveals that deep learning-based rotation invariant methods are effective for relatively easy settings with easy-to-distinguish pairs. However, their performance decreases significantly when the difference in rotations on the input pair is large, or when the degree of observation of input objects is reduced, or the difficulty level of input pair is increased. Finally, we connect feature encodings designed for rotation-invariant methods to 3D geometry that enable them to acquire the property of rotation invariance.
We propose a method that augments a simulated dataset using diffusion models to improve the performance of pedestrian detection in real-world data. The high cost of collecting and annotating data in the real-world has motivated the use of simulation platforms to create training datasets. While simulated data is inexpensive to collect and annotate, it unfortunately does not always closely match the distribution of real-world data, which is known as the sim2real gap. In this paper we propose a novel method of synthetic data creation meant to close the sim2real gap for the challenging pedestrian detection task. Our method uses a diffusion-based architecture to learn a real-world distribution which, once trained, is used to generate datasets. We mix this generated data with simulated data as a form of augmentation and show that training on a combination of generated and simulated data increases average precision by as much as 27.3% for pedestrian detection models in real-world data, compared against training on purely simulated data.
