Time series classification poses significant challenges due to the inherent temporal order of the data points and the existence of sequential dependencies between them. The ROCKET family, featuring methods like MiniROCKET, MultiROCKET, and HYDRA, is currently a leading approach in this domain, leveraging convolution kernels to aggregate temporal features into encodings for linear classifiers. However, these models encode temporal features over short temporal windows and then aggregate them as an unordered set of encodings over the longer temporal window of the entire data sequence. This prevents these models from capturing any longer sequence structure. To address this design drawback, we propose integrating hyperdimensional computing into ROCKET methods to explicitly incorporate temporal order of the short-term features within the entire time series. This approach enhances the discriminative power of encodings generated by MiniROCKET, MultiROCKET, and HYDRA where longer-term structure exists in the data, leading to increased classification performance with minimal computational overhead. More specifically, we introduce a method to represent time series as high-dimensional vectors through multiplicative binding of ROCKET encodings with encodings representing temporal order, applying this approach across various ROCKET methods. Additionally, we explore different high-dimensional vector representations of temporal order, yielding diverse similarity kernels that enhance classification accuracy. Through experiments on synthetic datasets, we highlight the limitations of ROCKET methods in handling temporal dependencies and show how the methods based on hyperdimensional computing overcome these limitations. Furthermore, our extensive experimental evaluation with real-world datasets included in the recent UCR archive, validates the advantages of our approach, consistently achieving classification improvements across all ROCKET methods that integrate hyperdimensional computing. Notably, our best model achieves a relative error rate reduction of over 50% compared to the best ROCKET model on several UCR datasets.

Hyperdimensional Computing (HDC) has shown promise in time series classification by enhancing MiniROCKET, forming HDC-MiniROCKET. However, the impact of choosing a specific HDC implementation, referred to as a Vector Symbolic Architecture (VSA), within this framework is so far unexplored. This paper systematically evaluates different VSAs within HDC-MiniROCKET, analyzing their impact on classification performance, hyperparameter sensitivity, and computational efficiency. Our findings reveal that certain VSAs require a significantly broader range of hyperparameter values to achieve optimal accuracy despite similar properties. We demonstrate this effect on a synthetic dataset and validate it across a subset of the real-world benchmark UCR, showing that VSA performance varies depending on the dataset characteristics. Additionally, we investigate the trade-off between computational complexity and classification accuracy. We find that computationally efficient VSAs not only reduce processing time but also achieve comparable or superior accuracy to more complex alternatives. These insights help to select VSAs in HDC-based time series classification.

Visual SLAM is a key technology for many autonomous systems. However, tracking loss can lead to the creation of disjoint submaps in multimap SLAM systems like ORB-SLAM3. Because of that, these systems employ submap merging strategies. As we show, these strategies are not always successful. In this paper, we investigate the impact of using modern VPR approaches for submap merging in visual SLAM. We argue that classical evaluation metrics are not sufficient to estimate the impact of a modern VPR component on the overall system. We show that naively replacing the VPR component does not leverage its full potential without requiring substantial interference to the original system. Because of that, we present a post-processing pipeline along with a set of metrics that allow us to estimate the impact of modern VPR components. We evaluate our approach on the NCLT and Newer College datasets using ORB-SLAM3 with the NetVLAD and HDC-DELF descriptors. Additionally, we present a simple approach for combining VPR with temporal consistency for map merging. We show that the map merging performance of ORB-SLAM3 can be improved. Building on these results, researchers in VPR can assess the potential of their approaches for SLAM systems.

The vector superposition operation plays a central role in Hyperdimensional Computing (HDC), enabling compositionality of hypervectors without expanding the dimensionality, unlike concatenation. However, a problem arises when the quantity of superimposed vectors surpasses a certain threshold, which is determined by the hypervector’s information capacity relative to its dimensionality. Beyond this point, cross-talk noise incrementally obscures the distinctiveness of individual hypervectors and information is lost. To solve this challenge, we introduce a novel method for weighting individual hypervectors within the superposition, ensuring that only those hypervectors crucial for a given task are prioritized. The weights are learned end-to-end using the backpropagation algorithm in a neural network. Our method is characterized by two key features: (1) The resultant weighting model is exceptionally compact, as the number of trainable weights is equal to the total number of hypervectors in the superposition; (2) The model offers enhanced explainability due to the compositional nature of its encoding. These features collectively contribute to the efficiency and effectiveness of our proposed classification approach using hyperdimensional computing. We illustrate our approach through the multi-channel time series classification task. In this framework, each channel is encoded as a hypervector-descriptor, and those are subsequently composed into a single hypervector via superposition. This superimposed vector forms the basis for training the classification model based on the neural network. Applying our approach of weighted superposition on this task improved the classification performance compared to standard superposition or concatenation of feature vectors, especially for larger numbers of channels.

Large-scale applications of Visual Place Recognition (VPR) require computationally efficient approaches. Further, a well-balanced combination of data-based and training-free approaches can decrease the required amount of training data and effort and can reduce the influence of distribution shifts between the training and application phases. This paper proposes a runtime and data-efficient hierarchical VPR pipeline that extends existing approaches and presents novel ideas. There are three main contributions: First, we propose Local Positional Graphs (LPG), a training-free and runtime-efficient approach to encode spatial context information of local image features. LPG can be combined with existing local feature detectors and descriptors and considerably improves the image-matching quality compared to existing techniques in our experiments. Second, we present Attentive Local SPED (ATLAS), an extension of our previous local features approach with an attention module that improves the feature quality while maintaining high data efficiency. The influence of the proposed modifications is evaluated in an extensive ablation study. Third, we present a hierarchical pipeline that exploits hyperdimensional computing to use the same local features as holistic HDC-descriptors for fast candidate selection and for candidate reranking. We combine all contributions in a runtime and data-efficient VPR pipeline that shows benefits over the state-of-the-art method Patch-NetVLAD on a large collection of standard place recognition datasets with 15$\%$ better performance in VPR accuracy, 54$\times$ faster feature comparison speed, and 55$\times$ less descriptor storage occupancy, making our method promising for real-world high-performance large-scale VPR in changing environments. Code will be made available with publication of this paper.

Class-incremental continual learning is an important area of research, as static deep learning methods fail to adapt to changing tasks and data distributions. In previous works, promising results were achieved using replay and compressed replay techniques. In the field of regular replay, GDumb [23] achieved outstanding results but requires a large amount of memory. This problem can be addressed by compressed replay techniques. The goal of this work is to evaluate compressed replay in the pipeline of GDumb. We propose FETCH, a two-stage compression approach. First, the samples from the continual datastream are encoded by the early layers of a pre-trained neural network. Second, the samples are compressed before being stored in the episodic memory. Following GDumb, the remaining classification head is trained from scratch using only the decompressed samples from the reply memory. We evaluate FETCH in different scenarios and show that this approach can increase accuracy on CIFAR10 and CIFAR100. In our experiments, simple compression methods (e.g., quantization of tensors) outperform deep autoencoders. In the future, FETCH could serve as a baseline for benchmarking compressed replay learning in constrained memory scenarios.

Consistent motion estimation is fundamental for all mobile autonomous systems. While this sounds like an easy task, often, it is not the case because of changing environmental conditions affecting odometry obtained from vision, Lidar, or the wheels themselves. Unsusceptible to challenging lighting and weather conditions, radar sensors are an obvious alternative. Usually, automotive radars return a sparse point cloud, representing the surroundings. Utilizing this information to motion estimation is challenging due to unstable and phantom measurements, which result in a high rate of outliers. We introduce a credible and robust probabilistic approach to estimate the ego-motion based on these challenging radar measurements; intended to be used within a loosely-coupled sensor fusion framework. Compared to existing solutions, evaluated on the popular nuScenes dataset and others, we show that our proposed algorithm is more credible while not depending on explicit correspondence calculation.

Classification of time series data is an important task for many application domains. One of the best existing methods for this task, in terms of accuracy and computation time, is MiniROCKET. In this work, we extend this approach to provide better global temporal encodings using hyperdimensional computing (HDC) mechanisms. HDC (also known as Vector Symbolic Architectures, VSA) is a general method to explicitly represent and process information in high-dimensional vectors. It has previously been used successfully in combination with deep neural networks and other signal processing algorithms. We argue that the internal high-dimensional representation of MiniROCKET is well suited to be complemented by the algebra of HDC. This leads to a more general formulation, HDC-MiniROCKET, where the original algorithm is only a special case. We will discuss and demonstrate that HDC-MiniROCKET can systematically overcome catastrophic failures of MiniROCKET on simple synthetic datasets. These results are confirmed by experiments on the 128 datasets from the UCR time series classification benchmark. The extension with HDC can achieve considerably better results on datasets with high temporal dependence at about the same computational effort for inference.

Robust and reliable online 3D multi-object tracking is an essential component of autonomous driving. Recent research follows the tracking-by-detection paradigm and focuses mainly on lidar sensors, due to their superior range, resolution and depth accuracy compared to other automotive sensors. This simplifies the challenging data association in crowded urban road scenes, resulting in a predominant status of laser based methods. In contrast, we propose an online 3D multi-object tracker based solely on mono camera images and radar data to promote non-lidar based tracking research. By representing all detections of one frame as a Gaussian mixture model (GMM), we are able to avoid a fixed data association, which may include wrong assumptions. Instead, we assign the GMM to each tracked object and solve the data association implicitly and jointly by estimating the full 3D object tracks in our factor graph based optimization back end. By including all available information from the object detector, our algorithm achieves accurate, robust and reliable tracking results. We conduct real world experiments on the nuScenes tracking data set improving the state-of-the-art for non-lidar based methods from 17.7% to 34.1 % AMOTA.

Place recognition is the task of recognizing the current scene from a database of known places.The currently dominant algorithmic paradigm is to use (deep learning based) holistic feature vectors to describe each place and use fast vector query methods to find matchings.We propose a novel type of image descriptor, Vector Semantic Representations (VSR), that encodes the spatial semantic layout from a semantic segmentation together with appearance properties in a, for example, 4,096 dimensional vector for place recognition.We leverage operations from the established class of Vector Symbolic Architectures to combine symbolic (e.g.class label) and numeric (e.g.feature map response) information in a common vector representation.We evaluate the proposed semantic descriptor on 13 standard mobile robotic place recognition datasets and compare to six descriptors from the literature.VSR is on par with the best compared descriptor (NetVLAD) in terms of mean average precision and superior in terms of recall and worst-case average precision.This makes the approach particularly interesting for candidate selection.For a more detailed investigation, we discuss and evaluate recall integrity as additional criterion.Further, we demonstrate that the semantic descriptor is particularly well suited for combination with existing appearance descriptors indicating that semantics provide complementary information for image matching.

Place recognition is the problem of finding associations between a query set of place descriptions and a database. It is an important means for loop closure detection in SLAM. The primary source of information to decide about associations is the pairwise similarity of descriptors between the query and the database items (e.g., image descriptor similarities). Beyond better descriptors, significant improvements were achieved by exploiting additional structural information, in particular by comparing sequences instead of individual items. In this letter, we propose to use another systematic source of information: intra-set similarities between items within the query or the database sets. They can be used to detect inconsistencies of groups of associations between database and query items, e.g. to inhibit matchings of multiple query descriptors to the same database descriptor if the query descriptors are mutually different. The underlying idea is a heuristic tightening of the triangle inequality of groups of descriptors. Based on a definition of matching inconsistencies, we propose an Inconsistency Resolution Procedure (IRP) to modify the inter-set similarities between database and query in a way that resolves existing inconsistencies with intra-set similarities. Our experiments show an average place recognition performance gain of >30% in a general place recognition setup with 21 datasets and two state of the art image processing front-ends. The proposed approach does not require additional information beyond descriptor similarities, makes no assumptions of sequences, does not require training, and has no parameter that needs adjustment. It can be combined with other established techniques like descriptor standardization and sequence processing.

Vector Symbolic Architectures combine a high-dimensional vector space with a set of carefully designed operators in order to perform symbolic computations with large numerical vectors. Major goals are the exploitation of their representational power and ability to deal with fuzziness and ambiguity. Over the past years, several VSA implementations have been proposed. The available implementations differ in the underlying vector space and the particular implementations of the VSA operators. This paper provides an overview of eleven available VSA implementations and discusses their commonalities and differences in the underlying vector space and operators. We create a taxonomy of available binding operations and show an important ramification for non self-inverse binding operations using an example from analogical reasoning. A main contribution is the experimental comparison of the available implementations in order to evaluate (1) the capacity of bundles, (2) the approximation quality of non-exact unbinding operations, (3) the influence of combining binding and bundling operations on the query answering performance, and (4) the performance on two example applications: visual place- and language-recognition. We expect this comparison and systematization to be relevant for development of VSAs, and to support the selection of an appropriate VSA for a particular task. The implementations are available.

Safe mobile robot navigation should consider not only collision avoidance with current obstacles but also include non-visible areas (to which we refer as blind spots) and the resulting risk of collision with hidden moving objects (e.g. people). Such capability is important for mobile robots operating in environments shared with humans - for instance a shopping assistant robot in a supermarket. This work aims to extend an existing motion planner for mobile robots (the Time Elastic Band planner) by including blind spots. As a result, the final planner does not only consider static and visible dynamic obstacles, but handles blind spots, too. To identify such blind spots, we define and use critical corners that imply them. Hence, our contributions in this paper are creating a critical corner detector, which operates on laser scan data, and the extension of a factor-graph-based path planner. We evaluate the proposed method standalone and in our simulation environment of a supermarket. It can be seen that the implementation is capable of detecting and dealing with blind spots. Finally, we provide source code for both the detector and the planner extensions.

Visual place recognition is the task of finding matchings of images that show the same place in the world. Combinations of appearance changes (e.g. changing illumination or weather) and geometric changes (e.g. viewpoint changes or occlusions) challenge existing approaches. Learning-based local image feature pipelines are a promising approach to this type of problem. We present a novel attentive feature pooling method that can be used to train a CNN to jointly detect and describe local image features. It can be trained on small or moderately sized datasets with weak supervision in a classification training setup (e.g. we use a set of 24k images of publicly available web-camera images in our experiments). We propose to use a joint loss function that combines the cross-entropy loss for the classification task with a mean squared error in order to increase the repeatability of feature detections. We show how the approach can be integrated in a place recognition pipeline and run experiments on several standard place recognition datasets. Despite the small training dataset, we demonstrate a 15% improvement in the average performance compared to the best of a number of compared state-of-the-art approaches, and, probably more importantly, a 3x improvement in the worst-case performance. Open source code is available.

Gaussian mixtures are a powerful and widely used tool to model non-Gaussian estimation problems. They are able to describe measurement errors that follow arbitrary distributions and can represent ambiguity in assignment tasks like point set registration or tracking. However, using them with common least squares solvers is still difficult. Existing approaches are either approximations of the true mixture or prone to convergence issues due to their strong nonlinearity. We propose a novel least squares representation of a Gaussian mixture, which is an exact and almost linear model of the corresponding log-likelihood. Our approach provides an efficient, accurate and flexible model for many probabilistic estimation problems and can be used as cost function for least squares solvers. We demonstrate its superior performance in various Monte Carlo experiments, including different kinds of point set registration. Our implementation is available as open source code for the state-of-the-art solvers Ceres and GTSAM.

Visual place recognition is an important subproblem of mobile robot localization. Since it is a special case of image retrieval, the basic source of information is the pairwise similarity of image descriptors. However, the embedding of the image retrieval problem in this robotic task provides additional structure that can be exploited, e.g., spatio-temporal consistency. Several algorithms exist to exploit this structure, e.g., sequence processing approaches or descriptor standardization approaches for changing environments. In this letter, we propose a graph-based framework to systematically exploit different types of additional structure and information. The graphical model is used to formulate a non-linear least squares problem that can be optimized with standard tools. Beyond sequences and standardization, we propose the usage of intra-set similarities within the database and/or the query image set as additional source of information. If available, our approach also allows to seamlessly integrate additional knowledge about poses of database images. We evaluate the system on a variety of standard place recognition datasets and demonstrate performance improvements for a large number of different configurations including different sources of information, different types of constraints, and online or offline place recognition setups.

In this paper, we present a novel approach for driving style classification based on time series data. Instead of automatically learning the embedding vector for temporal representation of the input data with Recurrent Neural Networks, we propose a combination of Hyperdimensional Computing (HDC) for data representation in high-dimensional vectors and much simpler feed-forward neural networks. This approach provides three key advantages: first, instead of having a “black box” of Recurrent Neural Networks learning the temporal representation of the data, our approach allows to encode this temporal structure in high-dimensional vectors in a human-comprehensible way using the algebraic operations of HDC while only relying on feed-forward neural networks for the classification task. Second, we show that this combination is able to achieve at least similar and even slightly superior classification accuracy compared to state-of-the-art Long Short-Term Memory (LSTM)-based networks while significantly reducing training time and the necessary amount of data for successful learning. Third, our HDC-based data representation as well as the feed-forward neural network, allow implementation in the substrate of Spiking Neural Networks (SNNs). SNNs show promise to be orders of magnitude more energy-efficient than their rate-based counterparts while maintaining comparable prediction accuracy when being deployed on dedicated neuromorphic computing hardware, which could be an energy-efficient addition in future intelligent vehicles with tight restrictions regarding on-board computing and energy resources. We present a thorough analysis of our approach on a publicly available data set including a comparison with state-of-the-art reference models.

Visual place recognition is the task of recognizing same places of query images in a set of database images. It is important for loop closure detection in SLAM and candidate selection for global localization. Many approaches in the literature perform computationally inefficient full image comparisons between queries and all database images. There is still a lack of suited methods for efficient place recognition that allow a fast, sparse comparison of only the most promising image pairs without any loss in performance. While this is partially given by approximate nearest neighbor (ANN) based methods, they trade speed for precision and additional memory consumption, and many cannot find arbitrary numbers of matching database images in case of loops in the database. In this paper, we propose a novel fast sequence-based method for efficient place recognition that can be applied online. It uses relocalization to recover from sequence losses, and exploits usually available but often unused intra-database similarities for a potential detection of all matching database images for each query in case of loops or stops in the database. We performed extensive experimental evaluations over five datasets and 21 sequence combinations, and show that our method outperforms two state-of-the-art approaches and even full image comparisons in many cases, while providing a good tradeoff between performance and percentage of evaluated image pairs. Code is available 1 .

Visual place recognition is the task of finding same places in a set of database images for a given set of query images. This task becomes particularly challenging if the environmental condition changes between database and query, for example from day to night. In this paper, we build upon our recent work on graph optimization for place recognition, where a graph was used to model additional structural knowledge like sequences. A subsequent non-linear least squares optimization (NLSQ) improved the place recognition performance. While this approach achieves very high performance, it is quite slow and memory inefficient. This paper addresses the long runtime and the high memory usage in order to obtain the same or better place recognition performance faster on larger problems. We propose a novel graph optimization procedure that is based on Iterated Conditional Modes (ICM). In addition, we investigate a new cost function for an edge in the graph. Our novel ICM-based approach achieves 9.1msec maximum runtime per query, which is 260 x faster than the minimum runtime with NLSQ. Moreover, with ICM we can optimize problems that are not feasible with NLSQ on a full graph due to memory limitations. To demonstrate the superior performance of our ICM-based method, we provide extensive experimental evaluations with the essence of 987 precision-recall curves: Our proposed ICM-based method is compared to the NLSQ-based method as well as to six sequence-based approaches from the literature on 21 sequence combinations from five datasets with four different image descriptors. Our experiments show that our ICM-based method with sequence-exploitation not only improves the NLSQ-based performance by 10% on average while being 385 x faster and using more than 60 x less memory. It also significantly outperforms all six sequence-based methods from the literature by at least 32% on average with the NetVLAD descriptor while using comparable runtime and memory. Code is available online(1).