Maritime cargo inspection requires imaging technologies capable of detecting concealed threats within dense, sealed containers, a role for which Muon Scattering Tomography (MST) is well suited: it images their interior through the density-dependent deflection of naturally occurring cosmic muons. However, MST remains constrained by the scarcity of labeled scans and by a cosmic muon flux that is both low and stochastic. Anomaly detection algorithms must therefore be trained on simulations, yet operate on measured scans acquired under different conditions, a sim-to-real gap that remains a central obstacle to operational deployment. We present the first end-to-end anomaly detection framework for maritime MST, from physically consistent simulations to validation on real container scans from the SilentBorder demonstration campaign. The task is cast as an out-of-distribution problem: the framework learns the spatial configurations of benign cargo and flags threats as deviations in the reconstruction error space, remaining agnostic to threat type and geometry. An attention U-Net, trained exclusively on benign synthetic scenes, preserves small-scale scattering signatures through its skip connections, and contraband consequently persists in the pixel-wise reconstruction error instead of being absorbed into the reconstructed background. A scoring function, the Homogeneity Index (HI), suppresses spatially uniform cosmic-ray statistical noise while amplifying coherent anomaly signatures: where pixel-level metrics collapse under a change of cargo configuration, HI retains its discriminative power. We evaluate three training strategies across two distinct cargo configurations under operational one-hour scan times, and test the best model on real muon cargo scans. The results for the studied scenarios indicate that the sim-to-real gap can be bridged.

Inverse problems like computed tomography, optical inverse rendering, and muon tomography, amongst others, occur in a vast range of scientific, medical, and security applications and are usually solved with highly specific algorithms depending on the task. Approaching these problems from a physical perspective and reformulating them as a function of particle interactions, enables 3D scene reconstruction in a physically consistent manner across particle-mediated modalities. Recent developments in differentiable volumetric rendering and optical optimization techniques, such as Neural Radiance Fields, Gaussian Splatting, and Scene Representations Networks (SRN), have been used to demonstrate the feasibility of jointly estimating unknown geometry and material parameters of a 3D scene. Some works also show the feasibility of modeling refraction and multiple scattering of light using differentiable optimization. By approaching these problems from a physical perspective and reformulating them in terms of transport and interaction of radiation or particles we can formulate a unified forward model that maps unknown scene parameters to measurements. This way we enable physically consistent 3D reconstruction for interactions that can be modeled by emission–absorption, refraction, and limited multiple scattering. Directly incorporating these interactions into a differentiable pipeline captured by a parameterized observer, allows decoupling the optimization procedure from both, the specific type of interaction and the capture mechanism. We perform a first experimental validation of our method using simulated and experimental optical scans from different sensing devices. Lastly, we explore the inter-domain capability of the new reconstruction method to other inverse problems, including muon tomography imaging.

Three-dimensional (3D) reconstruction of ships is an important part of maritime monitoring, allowing improved visualization, inspection, and decision-making in real-world monitoring environments. However, most state-ofthe-art 3D reconstruction methods require multi-view supervision, annotated 3D ground truth, or are computationally intensive, making them impractical for real-time maritime deployment. In this work, we present an efficient pipeline for single-view 3D reconstruction of real ships by training entirely on synthetic data and requiring only a single view at inference. Our approach uses the Splatter Image network, which represents objects as sparse sets of 3D Gaussians for rapid and accurate reconstruction from single images. The model is first fine-tuned on synthetic ShapeNet vessels and further refined with a diverse custom dataset of 3D ships, bridging the domain gap between synthetic and real-world imagery. We integrate a state-of-the-art segmentation module based on YOLOv8 and custom preprocessing to ensure compatibility with the reconstruction network. Postprocessing steps include real-world scaling, centering, and orientation alignment, followed by georeferenced placement on an interactive web map using AIS metadata and homography-based mapping. Quantitative evaluation on synthetic validation data demonstrates strong reconstruction fidelity, while qualitative results on real maritime images from the ShipSG dataset confirm the potential for transfer to operational maritime settings. The final system provides interactive 3D inspection of real ships without requiring real-world 3D annotations. This pipeline provides an efficient, scalable solution for maritime monitoring and highlights a path toward real-time 3D ship visualization in practical applications. Interactive demo: https://dlr-mi.github.io/ship3d-demo/.

The tracking-by-detection paradigm is the mainstream in multi-object tracking, associating tracks to the predictions of an object detector. Although exhibiting uncertainty through a confidence score, these predictions do not capture the entire variability of the inference process. For safety and security critical applications like autonomous driving, surveillance, etc., knowing this predictive uncertainty is essential though. Therefore, we introduce, for the first time, a fast way to obtain the empirical predictive distribution during object detection and incorporate that knowledge in multi-object tracking. Our mechanism can easily be integrated into state-of-the-art trackers, enabling them to fully exploit the uncertainty in the detections. Additionally, novel association methods are introduced that leverage the proposed mechanism. We demonstrate the effectiveness of our contribution on a variety of benchmarks, such as MOT17, MOT20, DanceTrack, and KITTI.

Cosmic-ray tomography usually relies on measuring the scattering or transmission of muons produced within cosmic-ray air showers to reconstruct an examined volume of interest (VOI). During the traversing of a VOI, all air shower particles, including muons, interact with the matter within the VOI producing so-called secondary particles. The characteristics of the production of these particles contain additional information about the properties of the examined objects and their materials. However, this approach has not been fully realized practically. Hence, this work aims to study a novel technique to scan shipping containers by comparing and combining the complementary results from stand-alone secondary particles and muon scattering using simulated simplified scenes with a 1 m3 cube made out of five different materials located inside the container. The proposed approach for a statistical combination is based on a multi-step procedure centered around a clustering and segmentation algorithm. This ensures a consistent evaluation and comparison of the results before and after the combination focusing on dedicated properties of the reconstructed object. The findings of this work show a potential improvement over the results obtained solely through muon scattering due to the utilization of secondary particle information by applying this novel dual-channel cosmic-ray tomography analysis.

To enhance maritime situational awareness, real-time segmentation of small or distant ships from optical monitoring footage, poses significant performance challenges, especially on embedded systems. Efficient processing of full-resolution images is essential for precise small ship segmentation. In this paper, we introduce a framework that combines an optimized version of ScatYOLOv8+CBAM with a custom batch-processed Slicing Aided Hyper Inference (SAHI). This approach is aimed at efficient and accurate small ship segmentation, deployed on embedded systems, and is validated using a real-world maritime dataset (ShipSG). With our optimized ScatYOLOv8+CBAM, we substantially improve inference efficiency with a 36% faster inference speed compared to its predecessor in the lightest model size, without compromising segmentation accuracy. Additionally, the integration of batch-processed SAHI, despite an increase in computation time, improves the accuracy of small ship segmentation up to 11%, allowing more effective utilization of full-resolution imagery without compromising the computational resources of embedded platforms. Our findings set a new benchmark for embedded maritime monitoring and pave the way for future research to optimize real-time high-resolution processing in resource-constrained environments.

The 2 nd Workshop on Maritime Computer Vision (MaCVi) 2024 addresses maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicles (USV). Three challenges categories are considered: (i) UAV-based Maritime Object Tracking with Re-ideruification, (ii) USV-based Maritime Obstacle Segmentation and Detection, (iii) USV-based Maritime Boat Tracking. The USV-based Maritime Obstacle Segmentation and Detection features three sub-challenges, including a new embedded challenge addressing efficicent inference on real-world embedded devices. This report offers a comprehensive overview of the findings from the challenges. We provide both statistical and qualitative analyses, evaluating trends from over 195 submissions. All datasets, evaluation code, and the leaderboard are available to the public at https://macvi.org/workshop/macvi24.

The 2nd Workshop on Maritime Computer Vision (MaCVi) 2024 addresses maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicles (USV). Three challenges categories are considered: (i) UAV-based Maritime Object Tracking with Re-identification, (ii) USV-based Maritime Obstacle Segmentation and Detection, (iii) USV-based Maritime Boat Tracking. The USV-based Maritime Obstacle Segmentation and Detection features three sub-challenges, including a new embedded challenge addressing efficicent inference on real-world embedded devices. This report offers a comprehensive overview of the findings from the challenges. We provide both statistical and qualitative analyses, evaluating trends from over 195 submissions. All datasets, evaluation code, and the leaderboard are available to the public at https://macvi.org/workshop/macvi24.

Ship recognition and georeferencing using monitoring cameras are crucial to many applications in maritime situational awareness. Although deep learning algorithms are available for ship recognition tasks, there is a need for innovative approaches that attain higher precision rates irrespective of ship sizes, types, or physical hardware limitations. Furthermore, their deployment in maritime environments requires embedded systems capable of image processing, with balanced accuracy, reduced latency and low energy consumption. To achieve that, we build upon the foundations of the standard YOLOv8 and present a novel architecture that improves the segmentation and georeferencing of ships in the context of maritime awareness using a real-world dataset (ShipSG). Our architecture synergizes global and local features in the image for improved ship segmentation and georeferencing. The 2D scattering-transform enhances the YOLOv8 backbone by extracting global structural features from the image. The addition of convolutional block attention module (CBAM) in the head allows focusing on relevant spatial and channel-wise regions. We achieve mAP of 75.46%, comparable to larger YOLOv8 models at a much faster inference speed, 59.3 milliseconds per image, when deployed on the NVIDIA Jetson Xavier AGX as target embedded system. We applied the modified network to georeference the segmented ship masks, with a georeferencing distance error of 18 meters, which implies comparable georeferencing performance to non-embedded approaches.

The correctness and precision of particle physics simulation software, such as Geant4, is expected to yield results that closely align with real-world observations or well-established theoretical predictions. Notably, the accuracy of these simulated outcomes is contingent upon the software's capacity to encapsulate detailed attributes, including its prowess in generating or incorporating complex geometrical constructs. While the imperatives of precision and accuracy are essential in these simulations, the need to manually code highly detailed geometries emerges as a salient bottleneck in developing software-driven physics simulations. This research proposes Blender-to-Geant4 (B2G4), a modular data workflow that utilizes Blender to create 3D scenes, which can be exported as geometry input for Geant4. B2G4 offers a range of tools to streamline the creation of simulation scenes with multiple complex geometries and realistic material properties. Here, we demonstrate the use of B2G4 in a muon scattering tomography application to image the interior of a sealed steel structure. The modularity of B2G4 paves the way for the designed scenes and tools to be embedded not only in Geant4, but in other scientific applications or simulation software.

Generative adversarial networks (GANs) are successfully used for image synthesis but are known to face instability during training. In contrast, probabilistic diffusion models (DMs) are stable and generate high-quality images, at the cost of an expensive sampling procedure. In this paper, we introduce a simple method to allow GANs to stably converge to their theoretical optimum, while bringing in the denoising machinery from DMs. These models are combined into a simpler model (ATME) that only requires a forward pass during inference, making predictions cheaper and more accurate than DMs and popular GANs. ATME breaks an information asymmetry existing in most GAN models in which the discriminator has spatial knowledge of where the generator is failing. To restore the information symmetry, the generator is endowed with knowledge of the entropic state of the discriminator, which is leveraged to allow the adversarial game to converge towards equilibrium. We demonstrate the power of our method in several image-to-image translation tasks, showing superior performance than state-of-the-art methods at a lesser cost. Code is available at https://github.com/DLR-MI/atme.