Large-language-model-based AI agents that combine retr-ieval, tool invocation, and generation are often evaluated with aggregate end-to-end scores that provide limited diagnostic information about the source of failures. This is bad practice for agent development because it hinders targeted improvements, prolongs development times, and may let errors slip through. We present a component-wise testing methodology in which system obligations are converted into automated, ground-truth-based tests before open-ended expert evaluation. The protocol separately tests whether required evidence is retrieved, whether the agent selects, orders, and parameterizes tools correctly, and whether the complete workflow solves closed-form integration tasks with exact answers. As a proof of concept, we instantiate the protocol in an agentic cardiology question-answering system that combines guideline retrieval, patient-data access, and computational tools. The resulting component tests show that curated semantic knowledge items improve Hit@5 retrieval by 15–20 percentage points over syntactic chunks, that model choice materially affects tool-use behavior, and that high accuracy on a narrow closed-form task can coexist with weaker tool-use performance across more diverse scenarios. Overall, this work argues that component-wise testing supports development-time evaluation by using automated, verifiable checks to localize failures, guide design choices, and provide practical reliability gates before broader expert evaluation and clinical validation. It should be adopted as a best practice in agent engineering.
Convolutional neural networks (CNNs) have revolutionized motor decoding from electroencephalographic (EEG) signals, showcasing their ability to outperform traditional machine learning, especially for Brain-Computer Interface (BCI) applications. By processing also other recording modalities (e.g., electromyography, EMG) together with EEG signals, motor decoding improved. However, multi-modal algorithms for decoding hand movements are mainly applied to simple movements (e.g., wrist flexion/extension), while their adoption for decoding complex movements (e.g., different grip types) is still under-investigated. In this study, we recorded EEG and EMG signals from 12 participants while they performed a delayed reach-to-grasping task towards one out of four possible objects (a handle, a pin, a card, and a ball), and we addressed multi-modal EEG+EMG decoding with a dual-branch CNN. Each branch of the CNN was based on EEGNet. The performance of the multi-modal approach was compared to mono-modal baselines (based on EEG or EMG only). The multi-modal EEG+EMG pipeline outperformed the EEG-based pipeline during movement initiation, while it outperformed the EMG-based pipeline in motor preparation. Finally, the multi-modal approach was capable of accurately discriminating between grip types widely during the task, especially from movement initiation. Our results further validate multi-modal decoding for potential future BCI applications, aiming at achieving a more natural user experience.
Graph matching aims at evaluating the dissimilarity of two graphs by defining a constrained correspondence between their nodes and edges. Error-tolerant graph matching, for instance, introduces the concept of a cost for penalizing structural differences in the matching. A popular method for this approach is graph edit distance, which is based on the cost of the minimal sequence of edit operations to transform a source graph into a target graph. One of the main problems of graph edit distance is the computational complexity, which is exponential in its exact form. In recent years, several approximation methods for graph edit distance have been presented which offer polynomial runtimes. In this paper, we approach the graph edit distance problem in a fundamentally different way. In particular, we propose to learn graph edit distance by means of graph neural networks. In a comprehensive experimental evaluation on six data sets, we verify that our approach not only provides comparable classification performance but also substantially reduces the runtime compared to a prominent algorithm for approximate graph edit distance computation.
River water temperature modeling is a major task in climate research. State-of-the-art methods for water temperature modeling deploy a transductive design, which makes it difficult to generalize to unseen water stations during test time. In the present paper, we isolate one of the common building blocks – a central LSTM, trained for each water station – and propose an embedding scheme in order to increase both the prediction accuracy and the amount of shared parameters and thus the generalization. The proposed embeddings are learned during training time. In an empirical evaluation we show that our method is able to reduce the RMSE by about five percentage points compared to the state-of-the-art reference method while decreasing the tuneable parameters by several orders of magnitude. We also provide a sample analysis of the embedding space of the catchment area of one specific river. Looking at the results of this qualitative analysis, we come to the conclusion that deploying an embedding in water temperature models is not only convincing to decrease the RMSE of water temperature predictions, but also enables better explainable deep learning models. Moreover, the proposed embedding technique opens up various unexplored applications in water temperature research.
Density estimation is crucial to statistical pattern recognition, in both the supervised and unsupervised frameworks. It is still an open problem, due to its intrinsic difficulties and to the many shortcomings of statistical parametric and non-parametric techniques. Artificial neural networks (ANN) have long been applied to the estimation of posterior probabilities for pattern classification (a simple supervised learning task), yet only a few attempts have been made to devise ANN-based density estimation algorithms. The paper proposes a novel algorithm for training an ANN from an unlabeled data sample of patterns randomly drawn from an underlying probability density function (PDF), say p(· ) , such that the ANN learns a robust non-parametric model of p(· ) . The algorithm leverages both the generalization capabilities of ANNs and the generality of the maximum-likelihood estimates of the parameters of Gaussian mixture models. Therefore, the proposed machine is termed Gaussian-mixture Neural Network (GNN). The best selling points of the GNN lie in its simplicity and effectiveness. Preliminary experimental results are reported and analyzed that involve data samples of variable size randomly drawn from PDFs of known form, either unimodal or multimodal. The GNN favorably compares with established statistical and ANN-based estimators, scoring generally higher than its competitors over a range of evaluation metrics. The code used in the experiments is made publicly available online on GitHub.
