Deep neural networks transform input data into latent representations that support a wide range of downstream tasks. These representations can be characterized along information-theoretic and geometric dimensions, but their relationship remains poorly understood. A central open question is whether low mutual information (MI) between inputs and representations necessarily implies geometrically compressed latent spaces and vice versa. We investigate this question using class-wise clustering as a measure of geometric compression and theoretically sound MI estimation in conditional entropy bottleneck (CEB) networks and continuous dropout networks. We evaluate the interplay between MI, geometric compression, and generalization on classification tasks under controlled noise injection schemes. Our findings show that low MI does not reliably correspond to geometric compression, and that the connection between the two is more nuanced than often assumed. Indeed, our experiments reveal a negative and nonlinear relationship that can reverse when varying training setup. Our results put forward a hypothesis that generalization acts as a potential confounder in this connection rather than being their direct consequence. Dataset: https://doi.org/10.17877/RCTRUST-2026-DWMJTZ . Code: https://github.com/link-er/information_geometric_compression .

Information plane (IP) analysis has been suggested to study the training dynamics of deep neural networks through mutual information (MI) between inputs, representations, and targets. However, its statistical validity is often compromised by the difficulty of estimating MI from samples of high-dimensional, deterministic representations. In this work, we perform IP analyses on binary neural networks (BNNs) where activations are discrete and MI is finite. We characterise the finite-sample behaviour of the plug-in entropy estimator and identify regimes for sample size N and representation dimensionality D under which MI estimates are reliable. Outside these regimes, we show that empirical MI estimates saturate to log_2 N, rendering IP trajectories uninformative. Restricting attention to the reliable regime, we train 375 BNNs to investigate the existence of late-stage compression phases and the relationship between compressed representations and generalisation performance. Our results show that while late-stage compression is frequently observed, compressed latent representations do not consistently correlate with improved generalization performance. Instead, the relationship between compression and generalisation is highly dependent on task, architecture, and regularisation.

Universal outlier hypothesis testing refers to a hypothesis testing problem where one observes a large number of length-n sequences – the majority of which are distributed according to the typical distribution π and a small number are distributed according to the outlier distribution μ – and one wishes to decide, which of these sequences are outliers without having knowledge of π and μ. In contrast to previous works, in this paper it is assumed that both the number of observation sequences and the number of outlier sequences grow with the sequence length. In this case, the typical distribution π can be estimated by computing the mean over all observation sequences, provided that the number of outlier sequences is sublinear in the total number of sequences. It is demonstrated that, in this case, one can achieve the error exponent of the maximum likelihood test that has access to both π and μ. However, this mean-based test performs poorly when the number of outlier sequences is proportional to the total number of sequences. For this case, a median-based test is proposed that estimates π as the median of all observation sequences. It is demonstrated that the median-based test achieves again the error exponent of the maximum likelihood test that has access to both π and μ, but only with probability approaching one. To formalize this case, the typical error exponent – similar to the typical random coding exponent introduced in the context of random coding for channel coding – is proposed.

We survey information-theoretic approaches to the reduction of Markov chains. Our survey is structured in two parts: The first part considers Markov chain coarse graining, which focuses on projecting the Markov chain to a process on a smaller state space that is informativeabout certain quantities of interest. The second part considers Markov chain model reduction, which focuses on replacing the original Markov model by a simplified one that yields similar behavior as the original Markov model. We discuss the practical relevance of both approaches in the field of knowledge discovery and data mining by formulating problems of unsupervised machine learning as reduction problems of Markov chains. Finally, we briefly discuss the concept of lumpability, the phenomenon when a coarse graining yields a reduced Markov model.

This paper employs physics-informed neural networks (PINNs) to solve Fisher's equation, a fundamental reaction-diffusion system with both simplicity and significance. The focus is on investigating Fisher's equation under conditions of large reaction rate coefficients, where solutions exhibit steep traveling waves that often present challenges for traditional numerical methods. To address these challenges, a residual weighting scheme is introduced in the network training to mitigate the difficulties associated with standard PINN approaches. Additionally, a specialized network architecture designed to capture traveling wave solutions is explored. The paper also assesses the ability of PINNs to approximate a family of solutions by generalizing across multiple reaction rate coefficients. The proposed method demonstrates high effectiveness in solving Fisher's equation with large reaction rate coefficients and shows promise for meshfree solutions of generalized reaction-diffusion systems.

Training physics-informed neural networks (PINNs) for forward problems often suffers from severe convergence issues, hindering the propagation of information from regions where the desired solution is well-defined. Haitsiukevich and Ilin (2023) proposed an ensemble approach that extends the active training domain of each PINN based on i) ensemble consensus and ii) vicinity to (pseudo-)labeled points, thus ensuring that the information from the initial condition successfully propagates to the interior of the computational domain. In this work, we suggest replacing the ensemble by a Bayesian PINN, and consensus by an evaluation of the PINN's posterior variance. Our experiments show that this mathematically principled approach outperforms the ensemble on a set of benchmark problems and is competitive with PINN ensembles trained with combinations of Adam and LBFGS.

When time series data contain missing values, it is common practice to substitute them using missing value imputation. However, if there is an unobserved abrupt change in the missing values, then standard imputation techniques are insufficient since they are biased towards normal data. Likewise, standard detectors cannot find abrupt changes that "hide" in missing data. To address these shortcomings, we propose Interval Forecast Imputation (IFI), which is a simple and intuitive combination of uncertainty intervals, forecasting, and anomaly detection that detects abrupt changes in missing time series data. A further advantage of IFI is that it is compatible with every state of the art forecasting technique-ranging from simple exponential smoothing over neural network-assisted forecasts to the popular Prophet library-while requiring only (9(1) additional time and space. In our experiments, we observe that IFI can detect abrupt changes in missing data and improves the imputation accuracy of all forecasting methods it is combined with.

It was recently shown that the loss function used for training physics-informed neural networks (PINNs) exhibits local minima at solutions corresponding to fixed points of dynamical systems. In the forward setting, where the PINN is trained to solve initial value problems, these local minima can interfere with training and potentially leading to physically incorrect solutions. Building on stability theory, this paper proposes a regularization scheme that penalizes solutions corresponding to unstable fixed points. Experimental results on four dynamical systems, including the Lotka-Volterra model and the van der Pol oscillator, show that our scheme helps avoiding physically incorrect solutions and substantially improves the training success rate of PINNs.

Ensuring trustworthiness in machine learning-by balancing utility, fairness, and privacy-remains a critical challenge, particularly in representation learning. In this work, we investigate a family of closely related information-theoretic objectives, including information funnels and bottlenecks, designed to extract invariant representations from data. We introduce the Conditional Privacy Funnel with Side-information (CPFSI), a novel formulation within this family, applicable in both fully and semi-supervised settings. Given the intractability of these objectives, we derive neural-network-based approximations via amortized variational inference. We systematically analyze the trade-offs between utility, invariance, and representation fidelity, offering new insights into the Pareto frontiers of these methods. Our results demonstrate that CPFSI effectively balances these competing objectives and frequently outperforms existing approaches. Furthermore, we show that by intervening on sensitive attributes in CPFSI's predictive posterior enhances fairness while maintaining predictive performance. Finally, we focus on the real-world applicability of these approaches, particularly for learning robust and fair representations from tabular datasets in data scarce-environments-a modality where these methods are often especially relevant.

Highly performing speech recognition is important for more fluent human-machine interaction (e.g., dialogue systems). Modern ASR architectures achieve human-level recognition performance on read speech but still perform sub-par on conversational speech, which arguably is or, at least, will be instrumental for human-machine interaction. Understanding the factors behind this shortcoming of modern ASR systems may suggest directions for improving them. In this work, we compare the performances of HMM- vs. transformer-based ASR architectures on a corpus of Austrian German conversational speech. Specifically, we investigate how strongly utterance length, prosody, pronunciation, and utterance complexity as measured by perplexity affect different ASR architectures. Among other findings, we observe that single-word utterances - which are characteristic of conversational speech and constitute roughly 30% of the corpus - are recognized more accurately if their F0 contour is flat; for longer utterances, the effects of the F0 contour tend to be weaker. We further find that zero-shot systems require longer utterance lengths and are less robust to pronunciation variation, which indicates that pronunciation lexicons and fine-tuning on the respective corpus are essential ingredients for the successful recognition of conversational speech.

Learning invariant representations that remain useful for a downstream task is still a key challenge in machine learning. We investigate a set of related information funnels and bottleneck problems that claim to learn invariant representations from the data. We also propose a new element to this family of information-theoretic objectives: The Conditional Privacy Funnel with Side Information, which we investigate in fully and semi-supervised settings. Given the generally intractable objectives, we derive tractable approximations using amortized variational inference parameterized by neural networks and study the intrinsic trade-offs of these objectives. We describe empirically the proposed approach and show that with a few labels it is possible to learn fair classifiers and generate useful representations approximately invariant to unwanted sources of variation. Furthermore, we provide insights about the applicability of these methods in real-world scenarios with ordinary tabular datasets when the data is scarce.

This research focuses on optimizing multi-stage manufacturing processes using Bayesian optimization (BO) with a robust Expected Improvement (EI) acquisition function. The aim is to optimize towards pre-selected target vectors, not to just minimize or maximize a function. To achieve this, we minimize the Euclidean distance between the actual and target output vectors, which requires transforming the Gaussian surrogate model posterior distribution into a non-central chi(2) ( NC chi(2) ) distribution. Furthermore, the distance measure additionally uses aleatoric uncertainty estimates of the actual output vectors to achieve robustness. We use a cascaded method that also considers the optimization results of intermediate stages, whereby optimization results are propagated from the last stage towards the first stage in each optimization iteration. By considering intermediate process outputs and aleatoric effects, our approach provides a robust optimization method for multi-stage manufacturing processes. To validate our method and to evaluate its properties, we use two artificial use cases. Moreover, we evaluate our approach in an industrial multi-stage forging process for the manufacturing of a nickel basis superalloy turbine disk, where involved stages are represented by corresponding 2D finite element method (FEM) DEFORM simulations. Evaluation suggests that our approach is superior in optimizing multi-stage manufacturing processes by considering all stage outcomes, robust distance measures, and the use of appropriate uncertainty distributions.

In this work, we investigate causal learning of independent causal mechanisms from a Bayesian perspective. Confirming previous claims from the literature, we show in a didactically accessible manner that unlabeled data (i.e., cause realizations) do not improve the estimation of the parameters defining the mechanism. Furthermore, we observe the importance of choosing an appropriate prior for the cause and mechanism parameters, respectively. Specifically, we show that a factorized prior results in a factorized posterior, which resonates with Janzing and Schölkopf's definition of independent causal mechanisms via the Kolmogorov complexity of the involved distributions and with the concept of parameter independence of Heckerman et al.

Launched on 2 June 2003 and arriving at Mars on 25 December 2003 after a 7-month interplanetary cruise, Mars Express was the European Space Agency’s first mission to arrive at another planet. After more than 20 years in orbit, the spacecraft and science payload remain in good health and the mission has become the second oldest operational planetary orbiter after Mars Odyssey. This contribution summarizes the Mars Express mission operations, science planning and data archiving systems, processes, and teams that are necessary to run the mission, plan the scientific observations, and execute all necessary commands. It also describes the data download, the ground processing and distribution to the scientific community for the study and analysis of Mars sub-surface, surface, atmosphere, magnetosphere, and moons. This manuscript also describes the main challenges throughout the history of the mission, including several potentially mission-ending anomalies. We summarize the evolution of the ground segment to provide new capabilities not envisaged before launch, whilst simultaneously maintaining or even increasing the quality and quantity of scientific data generated.

Conventional Radio Occultation measurements take place between a spacecraft in orbit around a planet (or during an occasional fly-by), and a receiving ground station on the Earth. When the straight line between the sender and receiver crosses the ionosphere and the atmosphere of the planet the signal is affected by the electrons in the ionosphere and the molecules in the neutral atmosphere. This signal change is observed as a slight frequency shift in the received radio signal with respect to the transmitted signal. The shift is very small compared to the Doppler shift in the frequency due to the differential relative velocity of the spacecraft and the Earth, but can for a stable system be isolated, and profiles of electron density in the ionosphere and profiles of atmospheric density in the neutral atmosphere can be derived.Here we are attempting to do occultation measurements between two spacecraft at Mars, Mars Express and ExoMars Trace Gas Orbiter, using the UHF link normally used for communicating between the orbiters and the landers on the surface of Mars, rather than using the X- or S-band transmitter used for the communication between a spacecraft and the Earth. This was first exercised between NASA’s Odyssey and MRO at three occasions in 2007 (Ao, et al., Radio Sci., 2015), but has not been repeated since then.Such a Mutual Occultation technique has several significant advantages over the traditional spacecraft to Earth occultation measurements. In particular, it allows a much more even distribution of the samples over the surface of the planet and in time, and the observations are independent of Earth occultation seasons. Due to the short period of the orbit of TGO many occultations take place every day. The distance between the two spacecraft is much smaller than the distance to the Earth and therefore a much higher signal to noise ratio can be achieved, resulting in very low statistical errors. The Mutual Occultation s/c to s/c link also avoids that the signal passes through the Earth’s ionosphere and atmosphere. The compensation for the Earth atmosphere and ionosphere is far from perfect and results in major contributions to the systematic errors. An additional positive effect is that the interplanetary plasma disturbance to the signal is a minimum due to the short distance between the two s/c and the large distance from the Sun.The first experiment was carried out on the 2nd of November 2020 and demonstrated the feasibility of the technique. Several more experiments took place during November-December. These initial test used a lander (Beagle2) hail signal from Mars Express/Melacom, consisting of a series of 20 seconds of modulated and unmodulated transmissions interspersed with 2 seconds without transmission. In March 2021 a software update was applied to Melacom, which allowed it to generate an uninterrupted, unmodulated carrier-only signal, since. This has improved the performance and the quality of the measurements significantly. To date four passes with this improved performance have been executed. These data are presently being analysed. This presentation will summarise recent simulations and show the latest results from these new experimental measurements. Example of Occultation Geometry: The orbits of TGO and MEX are indicated by tan respective red lines and the direct path of the radio signal is indicated by a white line. The part of the propagation through the atmosphere is indicated by a yellow box.

In this contribution we provide an update of the observations performed simultaneously (or quasi-simultaneously) by the two European satellites around Mars: Mars Express (MEX) and ExoMars 2016 Trace Gas Orbiter (TGO).The scientific objectives of both MEX and TGO missions are very complementary and there is a lot of synergy between the science teams, especially for the study of the atmosphere. Since the start of the TGO science operations in April 2018 we have executed hundreds of coordinated observations of the Martian atmosphere, first reported in [1] and now extended here. The science data obtained during these coordinated observations, both in sun occultation and nadir geometry, provide useful input for cross-calibration of the instruments, comparison of atmospheric vertical profiles and potential for the study of the temperature, composition, meteorology and climate of Mars.These observations are routinely coordinated by the instrument teams, in particular the spectrometers NOMAD and ACS onboard TGO [3,4] and SPICAM, OMEGA and PFS onboard MEX [5,6,7] and the Science Operations Centers (SOCs) of both missions [1, 2]:MEX-TGO Sun Occultations (within 15min and distance

In this contribution we provide an update of the observations performed simultaneously (or quasi-simultaneously) by the two European satellites around Mars: Mars Express (MEX) and ExoMars 2016 Trace Gas Orbiter (TGO).The scientific objectives of both MEX and TGO missions are very complementary and there is a lot of synergy between the science teams, especially for the study of the atmosphere. Since the start of the TGO science operations in April 2018 we have executed hundreds of coordinated observations of the Martian atmosphere, first reported in [1] and now extended here. The science data obtained during these coordinated observations, both in sun occultation and nadir geometry, provide useful input for cross-calibration of the instruments, comparison of atmospheric vertical profiles and potential for the study of the temperature, composition, meteorology and climate of Mars.These observations are routinely coordinated by the instrument teams, in particular the spectrometers NOMAD and ACS onboard TGO [3,4] and SPICAM, OMEGA and PFS onboard MEX [5,6,7] and the Science Operations Centers (SOCs) of both missions [1, 2]:MEX-TGO Sun Occultations (within 15min and distance

Artificial Neural Networks (ANNs) have emerged as a powerful tool in combustion simulations to replace memory-intensive tabulation of integrated chemical kinetics. Complex reaction mechanisms, however, present a challenge for standard ANN approaches as modeling multiple species typically suffers from inaccurate predictions on minor species. This paper presents a novel ANN approach which can be applied on complex reaction mechanisms in tabular data form, and only involves training a single ANN for a complete reaction mechanism. The approach incorporates a network architecture that automatically conserves mass and employs a particular loss weighting based on species depletion. Both modifications are used to improve the overall ANN performance and individual prediction accuracies, especially for minor species mass fractions. To validate its effectiveness, the approach is compared to standard ANNs in terms of performance and ANN complexity. Four distinct reaction mechanisms (H2, C7H16, C12H26, OME34) are used as a test cases, and results demonstrate that considerable improvements can be achieved by applying both modifications.

We propose the novel concept of anomaly-free regions (AFR) to improve anomaly detection. An AFR is a region in the data space for which it is known that there are no anomalies inside it, e.g., via domain knowledge. This region can contain any number of normal data points and can be anywhere in the data space. AFRs have the key advantage that they constrain the estimation of the distribution of non-anomalies: The estimated probability mass inside the AFR must be consistent with the number of normal data points inside the AFR. Based on this insight, we provide a solid theoretical foundation and a reference implementation of anomaly detection using AFRs. Our empirical results confirm that anomaly detection constrained via AFRs improves upon unconstrained anomaly detection. Specifically, we show that, when equipped with an estimated AFR, an efficient algorithm based on random guessing becomes a strong baseline that several widely-used methods struggle to overcome. On a dataset with a ground-truth AFR available, the current state of the art is outperformed.

Humanitarian organizations face a rising number of people fleeing violence or persecution, people who need their protection and support. When this support is given in the right locations, it can be timely, effective and cost-efficient. Successful refugee settlement planning not only considers the support needs of displaced people, but also local environmental conditions and available resources for ensuring survival and health. It is indeed very challenging to find optimal locations for establishing a new refugee camp that satisfy all these objectives. In this paper, we present a novel formulation of the facility location problem with a simulation-based evolutionary many-objective optimization approach to address this problem. We show how this approach, applied to migration simulations, can inform camp selection decisions by demonstrating it for a recent conflict in South Sudan. Our approach may be applicable to diverse humanitarian contexts, and the experimental results have shown it is capable of providing a set of solutions that effectively balance up to five objectives.