Monitoring cavity-nesting wild bees and wasps is vital for biodiversity research and conservation. Layer trap nests (LTNs) are emerging as a valuable tool to study the abundance and species richness of these insects, offering insights into their nesting activities and ecological needs. However, manually evaluating LTNs to detect and classify brood cells is labor-intensive and time-consuming. To address this, we propose a deep learning based approach for efficient brood cell detection and classification in LTNs. LTNs present additional challenges due to densely packed brood cells, leading to a high labeling effort per image. Moreover, we observe a significant imbalance in class distribution, with common species having notably more occurrences than rare species. Comprehensive labeling of common species is time-consuming and exacerbates data imbalance, while partial labeling introduces data incompleteness which degrades model performance. To reduce labeling effort and mitigate the impact of unlabeled data, we introduce a novel Constrained False Positive Loss (CFPL) strategy. CFPL dynamically masks predictions from unlabeled data, preventing them from interfering with the classification loss during training. We evaluate our approach on a dataset of 712 LTN images collected over one season, covering 28 fine-grained classes describing the taxonomy and status of brood cells. To minimize labeling effort, we limit the training set to a maximum of 300 labels per class. Experimental results demonstrate that deep learning can be effectively used to detect brood cells in LTNs. Our CFPL method further improves performance and balances model accuracy and labeling effort while also mitigating class imbalance.

Understanding the implications of climate change on ecosystems necessitates continuous monitoring of plant phenology. While citizen science data collected through smartphone applications offer a rich source of information, existing phenology studies predominantly focus on individual species. This study introduces a pioneering data science approach to quantify plant group behaviour from individual observations. Leveraging over ten million ground measurements of plant observations obtained through the Flora Incognita plant identification app in Germany from 2018 to 2023, our analysis unveils macrophenological patterns arising from plant group behaviour. The findings indicate nonlinear changes in group behaviour across the annual cycle. Furthermore, we explore the relationship between these macrophenological patterns on the ground and phenology derived from remote sensing data. The growing databases of citizen science holds great potential to investigate climate-induced phenological shifts and provide valuable insights into plant group behaviour.

Welding thin steel sheets in industrial applications is difficult because joint gaps occur during the process, which can lead to weld interruptions. Such welds are considered a reject and in order to avoid the weld to interrupt it is crucial to hinder the formation of joint gaps. Especially laser beam welding is affected by the emergence of gaps. Due to the narrow laser spot, product quality is highly dependent on the alignment and positioning of the sheets. This is typically done by clamping devices, which hold the workpieces in place. However, these clamps are suited for a specific workpiece geometry and require manual redesign every time the process changes. Adaptive clamping devices instead are designed to realize a time-dependent workpiece adjustment. Modeling the joint gap behavior to realize a controller for adaptive clamps can be difficult as the influence of heating, melting, and cooling on the joint gap formation is unknown and varies due to temperature dependent physical properties. Instead, the control parameters and actions can be derived using data-driven methods. In this paper, we present a novel data-driven approach how deep learning can be utilized to manipulate the sheet position during the weld with two actuators that apply force. A temporal convolution neural network (TCN) analyzes the change of the joint gap and predicts the required force to adapt the workpiece position. The developed method has been integrated into the welding process and improves the length of the average weld seam by 39.5% compared to welds without an active adjustment and 1.4% to welds that have been adapted with a constant force.

In the last twenty years, Echo State Networks have become a prominent method for the prediction of time series with a large variety of proposed topologies of connections. These topologies inside the ESN are typically shown to be advantageous for a certain prediction task in terms of reducing the computational complexity and increasing the explainability regarding the selection of weights. Still, a thorough comparison of these different topologies is missing in terms of their formal description and their performance as well as characteristics when applied to different prediction tasks. In this paper, we study 16 topologies for the task of time series prediction in the context of chaotic dynamics. We restrict our focus to those since they are considered among the most challenging to predict, while still being of interest for applications and being representative for other time series. We categorize the selected topologies into intralevel connections, concurrent and sequential reservoirs as well as complex topologies and implemented all of them. We parametrized all according to their original publications but also ran additional tests regarding suitable parametrization. All topologies are evaluated on the eight time series, in auto-regressive as well as single-step predictions. Our results emphasize on the benefit of choosing a suitable topology, reducing the error by up to 94.87% compared to the baseline, the conventional Echo State Network. We also compared the 16 topologies to the widely used Gated Recurrent Unit (GRU) network and observe the structured reservoir yielding a decreased error of up to 99.99% compared to the GRU.

Plant phenology investigates the timing of critical events in a plant's life cycle, encompassing budburst, flowering, fruiting, and senescence, with their significance rooted in their responsiveness to environmental conditions. Despite the growing interest in phenology, challenges persist in documenting these processes due to their extensive spatial and temporal scales. While global phenological networks traditionally collect data at the individual scale, a concern is arising regarding the declining number of phenological observers, prompting questions about the future of these datasets. Simultaneously, the surge in plant identification apps among citizens has yielded a substantial volume of plant occurrence records, accompanied by plant images, spanning diverse temporal and spatial scales. In this study, we explore the viability of utilizing opportunistically captured plant observations gathered through a plant identification app to determine the onset of flowering. Additionally, we investigate how citizen science-based phenological monitoring can be enhanced by incorporating images generated by the app. To achieve this, we developed a machine learning-based workflow enabling the automatic annotation of thousands of images into specific phenological stages. Beyond examining the onset of flowering, our established methodology allows for the exploration of other phenological stages, such as budburst or fruiting, on a large scale. Subsequently, we compare these opportunistic phenological records with systematically collected data from phenological networks. This approach not only streamlines image annotation but also augments the usefulness of citizen science data for phenological monitoring purposes.

Gradient Inversion (GI) attacks are a ubiquitous threat in Federated Learning (FL) as they exploit gradient leakage to reconstruct supposedly private training data. Common defense mechanisms such as Differential Privacy (DP) or stochastic Privacy Modules (PMs) introduce randomness during gradient computation to prevent such attacks. However, we pose that if an attacker effectively mimics a client's stochastic gradient computation, the attacker can circumvent the defense and reconstruct clients' private training data. This paper introduces several targeted GI attacks that leverage this principle to bypass common defense mechanisms. As a result, we demonstrate that no individual defense provides sufficient privacy protection. To address this issue, we propose to combine multiple defenses. We conduct an extensive ablation study to evaluate the influence of various combinations of defenses on privacy protection and model utility. We observe that only the combination of DP and a stochastic PM was sufficient to decrease the Attack Success Rate (ASR) from 100% to 0%, thus preserving privacy. Moreover, we found that this combination of defenses consistently achieves the best trade-off between privacy and model utility.

The prediction of time series is a challenging task relevant in such diverse applications as analyzing financial data, forecasting flow dynamics or understanding biological processes. Especially chaotic time series that depend on a long history pose an exceptionally difficult problem. While machine learning has shown to be a promising approach for predicting such time series, it either demands long training time and much training data when using deep Recurrent Neural Networks. Alternative, when using a Reservoir Computing approach it comes with high uncertainty and typically a high number of random initializations and extensive hyper-parameter tuning. In this paper, we focus on the Reservoir Computing approach and propose a new mapping of input data into the reservoir’s state space. Furthermore, we incorporate this method in two novel network architectures increasing parallelizability, depth and predictive capabilities of the neural network while reducing the dependence on randomness. For the evaluation, we approximate a set of time series from the Mackey-Glass equation, inhabiting non-chaotic as well as chaotic behavior as well as the SantaFe Laser dataset and compare our approaches in regard to their predictive capabilities to Echo State Networks, Autoencoder connected Echo State Networks and Gated Recurrent Units. For the chaotic time series, we observe an error reduction of up to 85.45% compared to Echo State Networks and 90.72% compared to Gated Recurrent Units. Furthermore, we also observe tremendous improvements for non-chaotic time series of up to 99.99% in contrast to the existing approaches.

Gradient Inversion (GI) attacks are a ubiquitous threat in Federated Learning as they exploit gradient leakage to reconstruct supposedly private training data. Recent work has proposed to prevent gradient leakage without loss of model utility by incorporating a PRivacy EnhanCing mODulE (PRECODE) based on variational modeling. Without further analysis, it was shown that PRECODE successfully protects against GI attacks. In this paper, we make multiple contributions. First, we investigate the effect of PRECODE on GI attacks to reveal its underlying working principle. We show that variational modeling introduces stochasticity into the gradients of PRECODE and the subsequent layers in a neural network. The stochastic gradients of these layers prevent iterative GI attacks from converging. Second, we formulate an attack that disables the privacy preserving effect of PRECODE by purposefully omitting stochastic gradients during attack optimization. To preserve the privacy preserving effect of PRECODE, our analysis reveals that variational modeling must be placed early in the network. However, early placement of PRECODE is typically not feasible due to reduced model utility and the exploding number of additional model parameters. Therefore, as a third contribution, we propose a novel privacy module—the Convolutional Variational Bottleneck (CVB)—that can be placed early in a neural network without suffering from these drawbacks. We conduct an extensive empirical study on three seminal model architectures and six image classification datasets. We find that all architectures are susceptible to GI attacks, which can be prevented by our proposed CVB. Compared to PRECODE, we show that our novel privacy module requires fewer trainable parameters, and thus computational and communication costs, to effectively preserve privacy.

LiDAR is crucial for robust 3D scene perception in autonomous driving. LiDAR perception has the largest body of literature after camera perception. However, multi-task learning across tasks like detection, segmentation, and motion estimation using LiDAR remains relatively unexplored, especially on automotive-grade embedded platforms. We present a real-time multi-task convolutional neural network for LiDAR-based object detection, semantics, and motion segmentation. The unified architecture comprises a shared encoder and task-specific decoders, enabling joint representation learning. We propose a novel Semantic Weighting and Guidance (SWAG) module to transfer semantic features for improved object detection selectively. Our heterogeneous training scheme combines diverse datasets and exploits complementary cues between tasks. The work provides the first embedded implementation unifying these key perception tasks from LiDAR point clouds achieving 3ms latency on the embedded NVIDIA Xavier platform. We achieve state-of-the-art results for two tasks, semantic and motion segmentation, and close to state-of-the-art performance for 3D object detection. By maximizing hardware efficiency and leveraging multi-task synergies, our method delivers an accurate and efficient solution tailored for real-world automated driving deployment. Qualitative results can be seen at https://youtu.be/H-hWRzv2lIY.

Additive manufacturing using the DED-Arc process is a costintensive process due to the long production times. Conventional quality testing procedures, which are carried out downstream of production, only provide information about the quality achieved once the component has been completed. Reworking is often impossible, meaning that the component is rejected if the quality is insufficient. By using in-situ quality controls, irregularities can be recognised at an early stage during the actual construction process and direct reworking can take place. Acoustic emissions are already being used successfully by experienced welders as a quality feature for the process. The aim of this work is to utilise the acoustic signals generated during welding to predict the process quality. The focus here is on detecting the shielding gas flow and any surface contamination caused by oil. This can contribute significantly to the formation of pores within the resulting weld bead. To recognise these parameters, the use of a Convolutional Neural Network (CNN) to evaluate the emitted airborne sound is discussed. The Mel Cepstral Coefficients (MFCC), which are widely used in speaker and speech recognition, are used as an essential feature for classification. In addition, the influence of the CNN's network parameters on the classification quality of the resulting network is shown. It is shown that the exclusive use of MFCC enables the CNN to detect process deviations. Due to the high information density of the MFCC compared to the STFT spectrum, the former offers the possibility of significantly reducing the size of the CNN used.

Opportunistic plant records provide a rapidly growing source of spatiotemporal plant observation data. Here, we used such data to explore the question whether they can be used to detect changes in species phenologies. Examining 19 herbaceous and one woody plant species in two consecutive years across Europe, we observed significant shifts in their flowering phenology, being more pronounced for spring-flowering species (6-17 days) compared to summer-flowering species (1-6 days). Moreover, we show that these data are suitable to model large-scale relationships such as "Hopkins' bioclimatic law" which quantifies the phenological delay with increasing elevation, latitude, and longitude. Here, we observe spatial shifts, ranging from -5 to 50 days per 1000 m elevation to latitudinal shifts ranging from -1 to 4 days per degree northwards, and longitudinal shifts ranging from -1 to 1 day per degree eastwards, depending on the species. Our findings show that the increasing volume of purely opportunistic plant observation data already provides reliable phenological information, and therewith can be used to support global, high-resolution phenology monitoring in the face of ongoing climate change.

Visual odometry is an ill-posed problem and utilized in many robotics applications, especially automated driving for mapless navigation. Recent applications have shown that deep models outperform traditional approaches especially in localization accuracy and furthermore significantly reduce catastrophic failures. The disadvantage of most of these models is a strong dependence on high-quantity and high-quality ground truth data. However, accurate and dense depth ground truth data for real world datasets is difficult to obtain. As a result, deep models are often trained on synthetic data which introduces a domain gap. We present a weakly supervised approach to overcome this limitation. Our approach uses estimated optical flow for training that can be generated without the need for high-quality dense depth ground truth. Instead, it only requires ground truth poses and raw camera images for training. In the experiments, we show that our approach enables deep visual odometry to be efficiently trained on the target domain (real data) while achieving state-of-the-art performance on the KITTI dataset.

Background: The Progress Test Medizin (PTM) is a formative test for medical students issued twice a year by the Charité-Universitätsmedizin Berlin. We used data from the PTM to automatically generate personalized feedback designed to identify the knowledge gaps of test participants as exactly as possible with the help of large language models and statistical analysis. Methods: We have developed a seven-step approach to fulfil the purpose of this study. Firstly, a large language model (ChatGPT 4.0) identified keywords in the form of MeSH terms from all 200 questions of one PTM run. These keywords were checked against the list of medical terms included in the MeSH thesaurus published by the National Library of Medicine (NLM). Meanwhile, answer patterns of PTM questions were also analysed to find whether, given a pair of questions (A,B), knowing the answer to A has a large effect on improving the chance of knowing the answer to B. If that is the case, we say that question A is a precursor question to question B. With this information, we obtained series of questions related to specific MeSH terms and used them to develop a framework that allowed us to assess the performance of PTM participants and compose personalized feedback structured around a curated list of medical topics. A clustering procedure was also applied to construct benchmarking sets. Results: We simulated the generation of personalized feedback for 1,401 PTM participants out of their test results, thereby producing specific information about their knowledge regarding a number of topics ranging from 34 to 243. Substantial knowledge gaps were found in 14.67% to 21.76% of rated learning topics, depending on the benchmarking set considered. Conclusion: We designed and tested a method to generate student feedback covering up to 243 medical topics defined by MeSH terms. The feedback received by students in the later stages of their studies was more detailed, as they tend to face more questions matching their knowledge level.

Zusammenfassung Die additive Fertigung mittels DED-Arc-Verfahren ist durch die hohen Fertigungszeiten ein kostenintensiver Prozess. Die klassischen Qualitätsprüfverfahren, die der Fertigung nachgelagert sind, geben erst nach Fertigstellung des Werkstückes Aufschluss über die erreichte Qualität. Nacharbeiten sind häufig ausgeschlossen, sodass bei unzureichender Qualität das Bauteil dem Ausschuss zugeführt wird. Die Anwendung von in situ Qualitätskontrollen ermöglicht es, Unregelmäßigkeiten frühzeitig bereits während des eigentlichen Aufbauprozess zu erkennen und eine direkte Nacharbeit kann stattfinden. Die akustischen Emissionen werden bereits erfolgreich von erfahrenen Schweißern als Qualitätsmerkmal für den Prozess eingesetzt. In diesem Beitrag soll mit Hilfe der akustischen Signale, die während des Schweißens entstehen, die Prozessqualität vorhergesagt werden. Dabei liegt der Fokus auf der Erkennung des Schutzgasflusses und eventuell vorhandener Oberflächenverunreinigung durch Öl. Dies kann maßgeblich zur Bildung von Poren innerhalb der entstehenden Schweißraupen beitragen. Zur Erkennung dieser Parameter wird die Verwendung eines Convolutional Neural Networks (CNN) zur Auswertung des emittierten Luftschalls diskutiert. Als wesentliches Merkmal zur Klassifizierung werden die, in der Sprecher- und Spracherkennung verbreiteten, Mel-Cepstralkoeffizienten (MFCC) herangezogen. Des Weiteren wird der Einfluss der Netzwerkparameter des CNN auf die Klassifizierungsgüte des resultierenden Netzwerkes dargestellt. Es zeigt sich, dass die ausschließliche Verwendung der MFCC dem CNN ermöglicht Prozessabweichungen zu detektieren. Aufgrund der hohen Informationsdichte der MFCC gegenüber dem STFT-Spektrum bieten erstere die Möglichkeit die Größe des verwendeten CNN erheblich zu reduzieren.

Exams like the formative Progress Test Medizin can enhance their effectiveness by offering feedback beyond numerical scores. Content-based feedback, which encompasses relevant information from exam questions, can be valuable for students by offering them insight into their performance on the current exam, as well as serving as study aids and tools for revision.Our goal was to utilize Large Language Models (LLMs) in preparing content-based feedback for the Progress Test Medizin and evaluate their effectiveness in this task.We utilize two popular LLMs and conduct a comparative assessment by performing textual similarity on the generated outputs. Furthermore, we study via a survey how medical practitioners and medical educators assess the capabilities of LLMs and perceive the usage of LLMs for the task of generating content-based feedback for PTM exams.Our findings show that both examined LLMs performed similarly. Both have their own advantages and disadvantages. Our survey results indicate that one LLM produces slightly better outputs; however, this comes at a cost since it is a paid service, while the other is free to use. Overall, medical practitioners and educators who participated in the survey find the generated feedback relevant and useful, and they are open to using LLMs for such tasks in the future.We conclude that while the content-based feedback generated by the LLM may not be perfect, it nevertheless can be considered a valuable addition to the numerical feedback currently provided.

Sequence-to-sequence models based on LSTM and GRU are a most popular choice for forecasting time series data reaching state-of-the-art performance. Training such models can be delicate though. The two most common training strategies within this context are teacher forcing (TF) and free running (FR). TF can be used to help the model to converge faster but may provoke an exposure bias issue due to a discrepancy between training and inference phase. FR helps to avoid this but does not necessarily lead to better results, since it tends to make the training slow and unstable instead. Scheduled sampling was the first approach tackling these issues by picking the best from both worlds and combining it into a curriculum learning (CL) strategy. Although scheduled sampling seems to be a convincing alternative to FR and TF, we found that, even if parametrized carefully, scheduled sampling may lead to premature termination of the training when applied for time series forecasting. To mitigate the problems of the above approaches we formalize CL strategies along the training as well as the training iteration scale. We propose several new curricula, and systematically evaluate their performance in two experimental sets. For our experiments, we utilize six datasets generated from prominent chaotic systems. We found that the newly proposed increasing training scale curricula with a probabilistic iteration scale curriculum consistently outperforms previous training strategies yielding an NRMSE improvement of up to 81% over FR or TF training. For some datasets we additionally observe a reduced number of training iterations. We observed that all models trained with the new curricula yield higher prediction stability allowing for longer prediction horizons.

Recent point cloud registration approaches often deal with a consecutive determination of coarse and fine feature correspondences for hierarchical pose refinement. Due to the unordered nature of point clouds, a common way to generate a subsampled representation for the coarse matching step is by applying 3D-sensitive convolution approaches. However, expensive grouping mechanisms such as nearest neighbour search have to be used to determine the associated fine features, generating individual associations for each point cloud and leading to an increased overall run-time. Furthermore current methods often tend to predict deficient point correspondences and rely on additional filtering by expensive registration backends like RANSAC impeding their application in time critical systems.To overcome these challenges, we present MagneticPillars utilizing a Birds-Eye-View (BEV) grid representation, entailing fixed affiliations between coarse and fine feature cells. We show that by extracting correspondences in this manner, a small amount of key points is already sufficient to achieve an accurate pose estimation without external optimization methods like RANSAC. We evaluate our approach on two autonomous driving datasets for the task of point cloud registration by applying SVD as the backend, where we outperform recent state-of-the-art methods, reducing the rotation and translation error by 12% and 40%, respectively, and to top it all off, cutting runtime in half.

Reconstructing the tree of life and understanding the relationships of taxa are core questions in evolutionary and systematic biology. The main advances in this field in the last decades were derived from molecular phylogenetics; however, for most species, molecular data are not available. Here, we explore the applicability of 2 deep learning methods-supervised classification approaches and unsupervised similarity learning-to infer organism relationships from specimen images. As a basis, we assembled an image data set covering 4144 bivalve species belonging to 74 families across all orders and subclasses of the extant Bivalvia, with molecular phylogenetic data being available for all families and a complete taxonomic hierarchy for all species. The suitability of this data set for deep learning experiments was evidenced by an ablation study resulting in almost 80% accuracy for identifications on the species level. Three sets of experiments were performed using our data set. First, we included taxonomic hierarchy and genetic distances in a supervised learning approach to obtain predictions on several taxonomic levels simultaneously. Here, we stimulated the model to consider features shared between closely related taxa to be more critical for their classification than features shared with distantly related taxa, imprinting phylogenetic and taxonomic affinities into the architecture and training procedure. Second, we used transfer learning and similarity learning approaches for zero-shot experiments to identify the higher-level taxonomic affinities of test species that the models had not been trained on. The models assigned the unknown species to their respective genera with approximately 48% and 67% accuracy. Lastly, we used unsupervised similarity learning to infer the relatedness of the images without prior knowledge of their taxonomic or phylogenetic affinities. The results clearly showed similarities between visual appearance and genetic relationships at the higher taxonomic levels. The correlation was 0.6 for the most species-rich subclass (Imparidentia), ranging from 0.5 to 0.7 for the orders with the most images. Overall, the correlation between visual similarity and genetic distances at the family level was 0.78. However, fine-grained reconstructions based on these observed correlations, such as sister-taxa relationships, require further work. Overall, our results broaden the applicability of automated taxon identification systems and provide a new avenue for estimating phylogenetic relationships from specimen images.

Federated learning (FL) enables collaborative and privacy-preserving training of machine learning (ML) models on federated data. However, the barriers to using FL are still high. First, collaboration procedures are use-case-specific and require manual preparation, setup, and configuration of execution environments. Second, establishing collaborations and matching collaborators is time-consuming due to heterogeneous intents as well as data properties and distributions. Third, debugging the process and keeping track of the artifacts created and used during collaboration is challenging. Our goal is to reduce these barriers by requiring as little technical knowledge from collaborators as possible. We contribute mechanisms for flexible collaboration composition and creation, automated collaborator matching, and provenance-based collaboration and artifact management.

Evolving software is a highly complex and creative problem in which a number of different strategies are used to solve the tasks at hand. These strategies and reoccurring coding patterns can offer insights into the process. However, they can be highly project or even task-specific. We aim to identify code change patterns in order to draw conclusions about the software development process. For this, we propose a novel way to calculate high-level file overarching diffs, and a novel way to parallelize pattern mining. In a study of 1000 Java projects, we mined and analyzed a total of 45,000 patterns. We present 13 patterns, showing extreme points of the 7 pattern categories we identified. We found that a large number of high-level change patterns exist and occur frequently. The majority of mined patterns were associated with a specific project and contributor, where and by whom it was more likely to be used. While a large number of different code change patterns are used, only a few, mostly unsurprising ones, are common under all circumstances. The majority of code change patterns are highly specific to different context factors that we further explore.