Vision foundation models, such as the Segment Anything Model (SAM), demonstrate strong zero-shot generalization but lack precision with anatomically challenging objects. In contrast, convolutional neural network (CNN)-based models, such as nnU-Net, achieve high accuracy on domain-specific data but struggle to generalize on unseen data. To address these complementary limitations, we propose an uncertainty-aware fusion framework that integrates the generalizability of foundation models with the anatomical precision of task-specific models for cardiac MRI segmentation. The proposed approach combines Dempster-Shafer Theory (DST) with an entropy-guided fallback mechanism to perform voxel-wise fusion of calibrated probability maps. Unlike simple ensemble methods, the framework takes into consideration the inter-model agreement, conflict, and the uncertainty of the models. DST fusion is applied where the models agree, while high-conflict regions are handled by an entropy-guided fallback mechanism that selects predictions from the more reliable model. Extensive evaluation on the M&Ms dataset (in-domain) and the ACDC dataset (cross-domain) demonstrates consistent improvements in Dice and IoU across model pairings of varying strengths. In-domain gains are modest, whereas cross-domain evaluations show substantially larger improvements. Notably, the nnU-Net+SAM2 pairing achieves relative gains of approximately 8% in Dice and 11% in IoU on the cross-domain dataset. Comparisons with simple averaging, ablation studies, and statistical analysis confirm the effectiveness of the proposed framework. To our knowledge, this is the first application of voxel-wise DST-based fusion to combine vision foundation models with task-specific CNNs for cardiac MRI segmentation.

Post-hoc saliency maps such as Grad-CAM are increasingly used to audit why a deployed vision model made a decision, yet the heatmap drifts when the input is rotated, even when the prediction is unchanged. In domains with no canonical orientation, such as histopathology and aerial imagery, this undermines using saliency as evidence. We ask whether that drift is faithful signal or noise introduced by the CAM operator, and answer it by measuring equivariance at every stage of the operator rather than inferring it from the network's output. The instability is not where one would guess: the channel weights are the most rotation-stable stage, and on ResNet-50 exactly stable, because a GAP+linear head makes the class gradient field spatially constant. What moves is the spatial activation tensor, and the classifier's own pooling discards that movement. A causal test confirms the consequence: occluding the pixels whose saliency drifts costs the model less than occluding random pixels, at either orientation. The drift is carried by degrees of freedom the classifier throws away, which is what makes removing it faithful rather than destructive. EquiGrad-CAM is a training-free wrapper that takes T rotated views, inverse-rotates each view's saliency into a common canonical frame, and averages. On the full ImageNet-1K validation set it raises equivariance over single-view Grad-CAM by +36.0

Recent works have shown that tackling offline reinforcement learning (RL) with a conditional policy produces promising results. Decision Transformers (DT) have shown promising results in offline reinforcement learning by leveraging sequence modeling. However, standard DT methods rely on return-to-go (RTG) tokens, which are heuristically defined and often suboptimal for goal-conditioned tasks. In this work, we introduce Quasimetric Decision Transformer (QuaD), a novel approach that replaces RTG with learned quasimetric distances, providing a more structured and theoretically grounded guidance signal for long-horizon decision-making. We explore two quasimetric formulations: interval quasimetric embeddings (IQE) and metric residual networks (MRN), and integrate them into DTs. Extensive evaluations on the AntMaze benchmark demonstrate that QuaD outperforms standard Decision Transformers, achieving state-of-the-art success rates and improved generalization to unseen goals. Our results suggest that quasimetric guidance is a viable alternative to RTG, opening new directions for learning structured distance representations in offline RL.

This paper presents a novel algorithm for hybrid landmark- and intensity-based registration of medical images. The model can be adjusted to relax the Thin Plate Spline landmark-matching constraint and cater to landmark uncertainties. Most importantly, the inclusion of the intensity term helps improve the overlap between the transformed template and reference images. The method will be applied to cardiac images in the presence of landmark localization or correspondence error and consistently yields suitable post-registration image overlaps.

The multi-objective Adam (MAdam) is a populationbased optimizer that integrates Adam exploitation with evolutionary algorithm exploration. The capability of the MAdam optimizer and its opposition-based extension (OMAdam) to provide a wide range of solutions was previously examined in training neural networks for binary classification. This study investigates the practical applications of these two optimizers in medical image classification, developing two novel diagnostic techniques based on the MNIST and real medical image datasets MedMNIST. For both optimizers, their functionality was demonstrated by generating a set of Pareto fronts comprised of corresponding model parameters in the decision space. In the first approach, false positives and false negatives in binary cross entropy were assigned as two conflicting loss functions. The optimized weight configuration of the corresponding singleobjective Adam optimizer was captured by the set of solutions along the spectrum of the Pareto fronts. MAdam and OMAdam optimizers achieve F1-scores comparable to those of the singleobjective Adam optimizer. In the second approach, by aligning conflicting performance metrics as differentiable loss functions in MAdam, the obtained Pareto front is a direct representation of the performance metrics space, including precision, sensitivity, and specificity.

Multiple imaging sensors are being used in surveillance, medical imaging, and machine vision. To effectively combine the information from multiple imaging sensors, image fusion techniques have emerged to produce an improved fused single image. This paper introduces a general framework for image fusion, incorporating the structural similarity index measure (SSIM), a highly effective measure of visual proximity aligned with the human visual system. To formulate image fusion, the SSIM-Mean is used for data fidelity of an optimization problem. To solve it numerically by gradient descent method an alternate expression of SSIM is proposed, utilizing Mean values and vector norms, alongside its derivative calculation. We apply the proposed method to the resolution enhancement of a set of given degraded images, where SSIM-Mean is used as data fidelity. The results will be evaluated both qualitatively and quantitatively. A significant advantage of the proposed formulation lies in its flexibility to accommodate various degradation operators and regularization terms, which marks a notable strength in this approach. Our code is available at https://github.com/fatemehhirbodvash/SSIM-Mean- .

In the field of many-objective optimization, obtaining a dense solution set is a challenging task, mostly due to having hyper-surface nature of Pareto-front; which cannot be covered by commonly utilized population sizes. This is particularly vital in scenarios where innovization and informed decision-making are crucial. The challenge stems from the constraints imposed by population size limitations in evolutionary algorithms, which impede the efficient exploration of multiple solutions. A contributing factor to this issue is the lack of long-term memory in the well-known evolutionary algorithms to retain these solutions. On the contrary, the effective training of machine learning-assisted optimization or innovization relies on a substantial amount of data, which can be provided by preserving these valuable solutions. Moreover, long-term memory can play a significant role in expensive many-objective optimization, where the repetition of the optimization process is both costly and time-consuming, similar to training deep neural networks. The study focuses on NSGA-III equipped with long-term memory and assessing its performance across 16 benchmark problems, encompassing DTLZ1 to DTLZ7 and WFG1 to WFG9, considering scenarios with 3, 5, and 10 objectives. This paper explores the benefits of incorporating long-term memory in terms of the ultimate optimization outcomes, including the number of non-dominated solutions, knee points, and Inverted Generational Distance (IGD).

Multi-loss functions are present in various aspects of deep learning. In multi-modal, cross-modal, and multi-task learning contexts, multi-loss functions are essential elements for handling complex data with diverse information sources. Different tasks or modalities may have conflicting objectives. By combining them into a single loss function, the model might struggle to strike the right balance between these objectives, leading to suboptimal performance. The Multi-objective Adam optimizer, also referred to as MAdam, is an extension of Adam optimizer that is applied for optimizing several competing loss functions in deep learning. The MAdam algorithm exhibits sensitivity to its initialization, necessitating the injection of ex-treme points into the initial population. Additionally, this scheme encounters difficulties in effectively capturing the disconnected and non-convex Pareto fronts. In this paper, an opposition-based scheme was introduced into MAdam framework as global search is necessary for escaping local optima in gradient-based multi-objective optimization approaches. The Opposition-based MAdam, explores multiple directions over the landscape, that leads to independence from specific initialization. In a series of experiments, we demonstrate the scalability of our method by capturing the entire Pareto front using the MNIST dataset for binary classification of digit images 2 and 3. This was achieved with a fully connected network, employing multi-objective mean absolute error and binary cross-entropy as losses. OMAdam matches Adam's Fl-score in the early generations, a result to its high exploratory capacity which enhances its performance in initial stages of classification tasks. This results in a reduction of computational costs compared to both Adam and MAdam. The variation in Fl-score values along the Pareto front trajectory enables practitioners to select a post hoc solution based on the trade-offs achieved among conflicting loss functions as multiple objectives. This contrasts with Adam, which offers limited options due to its single-solution approach.

The recent success of deep neural networks has generated remarkable growth in Artificial Intelligence (AI) research and has received much interest over the past few years. One of the main challenges for the broad adoption of deep learning-based models such as Convolutional Neural Networks (CNN) is the lack of understanding of their decisions. To address this issue, Explainable Artificial Intelligence (XAI) has been proposed to shift toward more transparent AI, resulting in the development of techniques to explain decisions by AI models. This paper aims to explore and develop a multi-scale scheme of LIME (Local Interpretable Model-Agnostic Explanations) applied to image classification to explain decisions made by CNN models through heatmaps of coarse to finer scales. More precisely, when LIME highlights large superpixels from a coarse scale, there may be smaller regions in the corresponding superpixel that influenced the model’s prediction at some finer scale. In the proposed multi-scale scheme, two weighting approaches, one based on Gaussian distribution and another parameter-free framework will be introduced to produce visual explanations observed from different scales. Promising results for multi-scale classification heatmaps of histopathology images are presented. More specifically, we investigated the proposed multi-scale approach on Camelyon16 dataset. The results show that the explanations are faithful to the underlying model, and the visualizations are reasonably interpretable.

Magnetic resonance imaging (MRI) is a common non-invasive imaging technique with high soft tissue contrast. Different MRI modalities are used for the diagnosis of various conditions including T1-weighted and T2-weighted MRI. In this paper, we introduce MTSR-MRI, a novel method that can not only upscale low-resolution scans but also translates between the T1-weighted and T2-weighted modalities. This will potentially reduce the scan time or repeat scans by taking low-resolution inputs in one modality and returning plausible high-resolution output in another modality. Due to the ambiguity that persists in image-to-image translation tasks, we consider the distribution of possible outputs in a conditional generative setting. The mapping is distilled in a low-dimensional latent distribution which can be randomly sampled at test time, thus allowing us to generate multiple plausible high-resolution outputs from a given low-resolution input. We validate the proposed method on the BraTS-18 dataset qualitatively and quantitatively using a variety of similarity measures. The implementation of this work will be available at https://github.com/AvirupJU/MTSR-MRI.

Many real-world optimization problems can be modelled with several competing objectives. Most of the time, such optimization problems fall under the category of expensive problems. These are problems in which each fitness evaluation is time-consuming, for example, one fitness call could take hours or even days to compute. The time-consuming process of evaluating the function values or gradient of the objective functions may degrade the running speed of many optimization algorithms. The coordinate search (CS) approach is introduced as a single-objective gradient-free method for addressing large-scale, non-convex, and costly optimization problems. Due to the low computation and memory requirements of the CS algorithm, it can also be efficiently extended to address multi-objective optimization problems. The subject of this study is to develop a CS-based algorithm aimed at computationally expensive multi-objective optimization problems. In order to generate a set of non-dominated solutions, a population is created to apply the CS algorithm on each individual and finally reach an optimized interval. We demonstrate the efficacy of the proposed multi-objective CS method by comparing it with NSGA-II and MOEA/D as one of the well-known multi-objective algorithms on ZDT benchmark functions. Promising results are reported with the assumption of a limited number of fitness evaluations (NFF) which is desired during tackling complex and expensive optimization problems. Another major advantage of the proposed algorithm is that it provides regions of the Pareto front, that using sampling can generate as many Pareto front solutions as needed unlike other common optimization algorithms including NSGA-II.

We apply a multi-agent Reinforcement Learning (RL) algorithm to single image super-resolution (SISR). In our novel implementation, each agent chooses a particular action from a fixed action set comprised of existing local enhancement operators to update each pixel intensity value. The pixel-wise arrangement of agents enables the algorithm to increase the resolution of an image by choosing optimal pixel intensity values from each option in a content-aware manner. While previous implementations of the model on SISR use Generative Adversarial Network (GAN)-based algorithms, we demonstrate that local operators can produce promising improved results without relying on the additional overhead of using machine learning techniques for the action space. Notably, we apply the proposed method to medical images, whereas previous implementations focused on natural images.

Multi-objective optimization is a prevalent challenge in the area of deep learning. There is a lack of robust multi-objective optimization methods applicable in deep learning capable of training networks by simultaneously optimizing conflicting multiple loss functions. Its applications include a wide range of deep neural network branches such as multi-loss, multi-task, multi-modal, and cross-modal learning. In this paper, we develop MAdam as a multi-objective extension of the well-known Adam optimization algorithm. MAdam is a classical population-based approach that uses the gradient information of multiple objectives to accelerate population convergence toward an optimal minimum. The method applied a non-dominated sorting algorithm to keep selective population members and improve the diversity across the landscape. The performance of MAdam is evaluated on the standard ZDT test functions as the proof of concept. Promising results show the capability of this approach to converge towards an estimated Pareto front and to generate a well-distributed set of non-dominated solutions.

For many practical applications in medical image analysis and computer-aided diagnosis (CAD), it is necessary to accurately capture intricate anatomical and pathological details, given imaging acquisitions in different modalities. We introduce a novel GAN (Generative Adversarial Network) transformer-based model designed for combined super-resolution and modality translation of magnetic resonance images (MRI). The model aims to improve clinical workflows by enhancing image resolution and translating between different imaging modalities, e.g., T1 and T2 MRI data, by offering more detailed visualization that could potentially aid diagnosis and treatment planning. The approach will be validated quantitatively and qualitatively on the publicly available BraTS imaging dataset to provide a 4x increase in resolution and modality translation between T1 and T2 MRI pairs to demonstrate its potential.

The Proximal Policy Optimization (PPO) is a policy gradient approach providing state-of-the-art performance in many domains through the “surrogate” objective function using stochastic gradient ascent. While PPO is an appealing approach in reinforcement learning, it does not consider the importance of states (a frequently seen state in a successful trajectory) in policy/value function updates. In this work, we introduce Preferential Proximal Policy Optimization (P3O) which incorporates the importance of these states into parameter updates. First, we determine the importance of each state based on the variance of the action probabilities given a particular state multiplied by the value function, normalized and smoothed using the Exponentially Weighted Moving Average. Then, we incorporate the state's importance in the surrogate objective function. That is, we redefine value and advantage estimation objectives functions in the PPO approach. Unlike other related approaches, we select the importance of states automatically which can be used for any algorithm utilizing a value function. Empirical evaluations across six Atari environments demonstrate that our approach significantly outperforms the baseline (vanilla PPO) across different tested environments, highlighting the value of our proposed method in learning complex environments.

Over the past few years deep learning-based techniques such as Generative Adversarial Networks (GANs) have significantly improved solutions to image super-resolution and image-to-image translation problems. In this paper, we propose a solution to the joint problem of image super-resolution and multi-modality image-to-image translation. The problem can be stated as the recovery of a high-resolution image in a modality, given a low-resolution observation of the same image in an alternative modality. Our paper offers two models to address this problem and will be evaluated on the recovery of high-resolution day images given low-resolution night images of the same scene. Promising qualitative and quantitative results will be presented for each model.

Spatial resolution of medical images can be improved using super-resolution methods. Real Enhanced Super Resolution Generative Adversarial Network (Real-ESRGAN) is one of the recent effective approaches utilized to produce higher resolution images, given input images of lower resolution. In this paper, we apply this method to enhance the spatial resolution of 2D MR images. In our proposed approach, we slightly modify the structure of the Real-ESRGAN to train 2D Magnetic Resonance images (MRI) taken from the Brain Tumor Segmentation Challenge (BraTS) 2018 dataset. The obtained results are validated qualitatively and quantitatively by computing SSIM (Structural Similarity Index Measure), NRMSE (Normalized Root Mean Square Error), MAE (Mean Absolute Error), and VIF (Visual Information Fidelity) values.

Focal cortical dysplasia (FCD) is one of the most common lesions responsible for drug-resistant epilepsy, and is frequently missed by visual inspection. FCD may be amenable to surgical resection to achieve seizure freedom. By improving lesion detection the surgical outcome of these patients can be improved. Image processing techniques are a potential tool to improve the detection of FCD prior to epilepsy surgery. In this research, we propose and compare the performance of two type of models, Fully Convolutional Network (FCN) and a multi-sequence FCN to classify and segment FCD in children with drug-resistant epilepsy. This experiment utilized the volumetric T1-weighted, T2 weighted and FLAIR sequences. The whole slice FCN models were applied to each sequence separately while the multi-sequence model leverages combined information of all three sequences simultaneously. A leave-one-subject-out technique was utilized to train and evaluate the models. We evaluated subjectwise sensitivity and specificity, which corresponds to the ability of the model to classify those with or without a lesion. We also evaluated lesional sensitivity and specificity, which expresses the ability of the model to segment the lesion and the dice coefficient to evaluate lesion coverage. Our data consisted of 80 FCD subjects (56 MR-positive and 24 MR-negative) and 15 healthy controls. Performance of whole slice FCN was best on T1-weighted, followed by T2-weighted and lowest with FLAIR sequences. Multi-sequence model performed better than the T1 whole slice FCN, and detected 98% vs. 93% respectively MR-positive cases, and 92% vs. 88% respectively MR-negative cases, as well as achieved lesion coverage of 74% vs. 67% respectively for MR-positive cases and 68% vs. 64% for MR negative cases. The dice coefficient for the multi-sequence model was 57% and for whole slice FCN was 56% for MR-positive cases. In the test cohort of six new cases, the multi-sequence model detected 4 out of 6 cases where the predicted lesion had 56% overlap with the actual lesion. This work showed that deep learning methods in particular fully convolutional networks are a promising tool for classification and segmentation of FCD. Additional work is required to further improve lesion classification and segmentation, particularly for small lesions, as well as to train and test optimal algorithms on a larger multi-center dataset.

Sentiment analysis is one of the crucial tasks in Natural Language Processing (NLP) which refers to classifying natural language sentences by their positive or negative sentiments. In many existing deep learning-based models, providing an explanation of a sentiment might be as necessary as the prediction itself. In this study, we use four different classification models applied to the sentiment analysis of the Internet Movie Database (IMDB) reviews, and investigate the explainability of results using Local Interpretable Model-agnostic Explanation (LIME). Our results reveal how the attention-based models, such as Bidirectional LSTM (BiLSTM) and fine-tuned Bidirectional Encoder Representations from Transformers (BERT) would focus on the most relevant keywords.