Interpreting the decisions of deep image classifiers remains challenging, particularly in black-box settings where model internals are inaccessible. We introduce OCCAM, a framework for open-set causal concept explanation and ontology induction in vision models. OCCAM discovers visual concepts in an open-set manner, localizes them via text-guided segmentation, and performs object-level interventions by removing concepts to measure changes in class confidence, estimating each concept's causal contribution. Beyond local explanations, OCCAM aggregates interventional evidence across a dataset to induce a structured concept ontology that captures how classifiers globally organize visual concepts. Reasoning over this ontology reveals consistent dependencies between concepts, exposes latent causal relations, and uncovers systematic model biases. Experiments on Broden and ImageNet-S across multiple classifiers show that OCCAM improves explanation quality in open-set black-box settings while providing richer global insight than per-image attribution methods.

Contrastively trained vision-language models such as CLIP provide strong zero-shot transfer by aligning images and text in a shared embedding space. However, adapting these models to downstream tasks without degrading their open-vocabulary generalization remains challenging. Existing parameter-efficient adaptation methods typically improve task specialization through learned prompts, adapters, or multimodal transformations, where adaptation capacity is primarily expressed through additional trainable parameters. Inspired by recent latent reasoning methods in language models, we investigate a complementary perspective: can adaptation emerge from iterative reasoning on latent representations rather than from increasing parameter count alone? We introduce PERL (Parameter-Efficient Reasoning in CLIP Latent Space), a lightweight adaptation framework that augments a frozen CLIP model with a compact shared reasoning module applied recurrently across refinement steps. At each step, PERL generates a latent reasoning token conditioned on the current representation and injects it into an intermediate encoder layer, progressively refining higher-level semantic representations while preserving CLIP's pretrained multimodal structure. Across 15 benchmarks spanning base-to-novel generalization, cross-dataset transfer, and out-of-distribution ImageNet variants, PERL achieves the best parameter-performance trade-off among the compared methods under a fast-adaptation few-shot setting, combining strong novel-class accuracy and competitive transfer performance with only about 6K trainable parameters, up to 817x fewer than the largest compared approach. Overall, our results suggest that iterative latent reasoning provides a complementary adaptation mechanism to parameter scaling in discriminative vision-language models.

Ensuring trustworthiness in open-world visual recognition requires models that are interpretable, fair, and robust to distribution shifts. Yet modern vision systems are increasingly deployed as proprietary black-box APIs, exposing only output probabilities and hiding architecture, parameters, gradients, and training data. This opacity prevents meaningful auditing, bias detection, and failure analysis. Existing explanation methods assume white- or gray-box access or knowledge of the training distribution, making them unusable in these real-world settings. We introduce UNBOX, a framework for class-wise model dissection under fully data-free, gradient-free, and backpropagation-free constraints. UNBOX leverages Large Language Models and text-to-image diffusion models to recast activation maximization as a purely semantic search driven by output probabilities. The method produces human-interpretable text descriptors that maximally activate each class, revealing the concepts a model has implicitly learned, the training distribution it reflects, and potential sources of bias. We evaluate UNBOX on ImageNet-1K, Waterbirds, and CelebA through semantic fidelity tests, visual-feature correlation analyses and slice-discovery auditing. Despite operating under the strictest black-box constraints, UNBOX performs competitively with state-of-the-art white-box interpretability methods. This demonstrates that meaningful insight into a model's internal reasoning can be recovered without any internal access, enabling more trustworthy and accountable visual recognition systems.

Two-photon calcium imaging presents a challenging setting for foundation models: image appearance varies substantially across recordings and experimental conditions, annotations are scarce, and rapid adaptation is often needed. Rather than adapting model weights through fine-tuning, we ask whether a foundation model can be guided at inference time by injecting external visual memory directly into its input. We implement this idea with SAM 3 and introduce Retrieval-Augmented Visual Prompting (RAVP), a framework in which each target tile is augmented with a retrieved annotated exemplar whose bounding box is used as a concept prompt. RAVP turns retrieval into a form of visual prompting and enables adaptation through input design alone. We study multiple exemplar selection strategies, including fluorescence-guided heuristics and a lightweight recall predictor trained to estimate which exemplar is most informative for a target tile. Experiments on the Allen Brain Observatory show that exemplar-augmented inference consistently strengthens zero-shot neuron detection and instance segmentation. Ablation studies further show that a single carefully selected exemplar is more effective than prompting with multiple retrieved examples. These results position inference-time visual memory injection as a simple and effective alternative to parameter adaptation for foundation models in specialized biomedical imaging.

Cognitive impairment is a growing public health concern, with early detection playing a crucial role in improving patient outcomes. The Montreal Cognitive Assessment (MoCA) is widely used for screening mild cognitive impairment (MCI) and early-stage dementia. However, traditional MoCA assessments require manual scoring by trained professionals, making the process labor-intensive, time-consuming, and susceptible to human error. To overcome these limitations, we propose an automated pipeline for MoCA score estimation using eye-gaze data and Vision Transformers (ViTs). Our approach leverages gaze-tracking technology to capture spatial and temporal eyemovement patterns during structured cognitive tasks, identifying subtle cognitive impairments that may otherwise go unnoticed. The raw gaze data is preprocessed and mapped onto taskrelevant image regions, where a pretrained ViT extracts highdimensional feature representations. To address inconsistencies in gaze sampling and improve temporal modeling, we introduce a time-aware positional embedding mechanism that enhances the model's ability to infer cognitive performance. These extracted features are then processed by a transformer-based classification model to predict MoCA scores with high accuracy. We validate our approach using a dataset collected from seven cognitive gaming sessions, demonstrating its effectiveness in automated cognitive assessment. The experimental results indicate that our method provides a reliable and efficient alternative to traditional MoCA evaluations, reducing dependency on human intervention while maintaining diagnostic accuracy.

The paper presents CTVTL-CLIP, a conditional visuo-textual prompt tuning method designed to adapt large mul-timodal models for highly specialized applications, where the required semantic representations are not initially encoded in the models' latent space. CTVTL-CLIP jointly optimizes CLIP's text and visual encoders using sets of soft prompts that condition the representations on domain-specific knowledge and features, with the support of a lightweight pretrained network to condition textual learnable tokens. This approach enhances the model's ability to accurately understand medical images, aligning textual descriptions with visual representations even in cases with limited annotated data. CTVTL-CLIP was tested on an endoscopy image dataset containing multiple gastric lesions, significantly outperforming traditional classifiers such as CNNs and vision transformers, while using fewer learnable parameters. It was also evaluated on two additional medical image analysis tasks, including skin lesion classification and stenosis classification in angiographies, showing improved performance compared to state-of-the-art methods that require training more parameters. The combination of CTVTL-CLIP's efficiency and superior performance under-scores its practical potential for real-world medical applications, where data scarcity and model efficiency are critical challenges.

As our understanding of emotions continues to evolve, the ability of machines to accurately interpret and respond to emotional cues is more important than ever. Traditional methods of emotion recognition often fall short, particularly when it comes to the subtle and complex responses elicited by music. The EEG-Music Emotion Recognition Challenge aims to leverage electroencephalography (EEG) to decode emotional states from brain signals while subjects listen to music. This initiative seeks to uncover the intricate relationship between neural activity and emotional responses, offering insights for advancing adaptive user interfaces. We propose two tracks: (1) Person Identification aims to identify the subject from whom the EEG was recorded, while (2) Emotion Recognition targets the decoding of emotional state of the subject while listening to a musical stimulus.

Understanding and explaining the behavior of machine learning models is essential for building transparent and trustworthy AI systems. We introduce DEXTER, a data-free framework that employs diffusion models and large language models to generate global, textual explanations of visual classifiers. DEXTER operates by optimizing text prompts to synthesize class-conditional images that strongly activate a target classifier. These synthetic samples are then used to elicit detailed natural language reports that describe class-specific decision patterns and biases. Unlike prior work, DEXTER enables natural language explanation about a classifier's decision process without access to training data or ground-truth labels. We demonstrate DEXTER's flexibility across three tasks—activation maximization, slice discovery and debiasing, and bias explanation—each illustrating its ability to uncover the internal mechanisms of visual classifiers. Quantitative and qualitative evaluations, including a user study, show that DEXTER produces accurate, interpretable outputs. Experiments on ImageNet, Waterbirds, CelebA, and FairFaces confirm that DEXTER outperforms existing approaches in global model explanation and class-level bias reporting. Code is available at https://github.com/perceivelab/dexter.

BACKGROUND:Despite evidence supporting use of fractional flow reserve (FFR) and instantaneous waves-free ratio (iFR) to improve outcome of patients undergoing coronary angiography (CA) and percutaneous coronary intervention, such techniques are still underused in clinical practice due to economic and logistic issues. OBJECTIVES:We aimed to develop an artificial intelligence (AI)-based application to compute FFR and iFR from plain CA. METHODS AND RESULTS:Consecutive patients performing FFR or iFR or both were enrolled. A specific multi-task deep network exploiting 2 projections of the coronary of interest from standard CA was appraised. Accuracy of prediction of FFR/iFR of the AI model was the primary endpoint, along with sensitivity and specificity. Prediction was tested both for continuous values and for dichotomous classification (positive/negative) for FFR or iFR. Subgroup analyses were performed for FFR and iFR.A total of 389 patients from 5 centers were enrolled. Mean age was 67.9 ± 9.6 and 39.2% of patients were admitted for acute coronary syndrome. Overall, the accuracy was 87.3% (81.2-93.4%), with a sensitivity of 82.4% (71.9-96.4%) and a specificity of 92.2% (90.4-93.9%). For FFR, accuracy was 84.8% (77.8-91.8%), with a sensitivity of 81.9% (69.4-94.4%) and a specificity of 87.7% (85.5-89.9%), while for iFR accuracy was 90.2% (86.0-94.6%), with a sensitivity of 87.2% (76.6-97.8%) and a specificity of 93.2% (91.7-94.7%, all confidence intervals 95%). CONCLUSION:The presented machine-learning based tool showed high accuracy in prediction of wire-based FFR and iFR.

We introduce SeeingSounds, a lightweight and modular framework for audio-to-image generation that leverages the interplay between audio, language, and vision-without requiring any paired audio-visual data or training on visual generative models. Rather than treating audio as a substitute for text or relying solely on audio-to-text mappings, our method performs dual alignment: audio is projected into a semantic language space via a frozen language encoder, and, contextually grounded into the visual domain using a vision-language model. This approach, inspired by cognitive neuroscience, reflects the natural cross-modal associations observed in human perception. The model operates on frozen diffusion backbones and trains only lightweight adapters, enabling efficient and scalable learning. Moreover, it supports fine-grained and interpretable control through procedural text prompt generation, where audio transformations (e.g., volume or pitch shifts) translate into descriptive prompts (e.g., "a distant thunder") that guide visual outputs. Extensive experiments across standard benchmarks confirm that SeeingSounds outperforms existing methods in both zero-shot and supervised settings, establishing a new state of the art in controllable audio-to-visual generation.

Speech segmentation at both word and phoneme levels is crucial for various speech processing tasks. It significantly aids in extracting meaningful units from an utterance, thus enabling the generation of discrete elements. In this work we propose a model-agnostic framework to perform word boundary detection in a supervised manner also employing a labels augmentation technique and an output-frame selection strategy. We trained and tested on the Buckeye dataset and only tested on TIMIT one, using state-of-the-art encoder models, including pre-trained solutions (Wav2Vec 2.0 and HuBERT), as well as convolutional and convolutional recurrent networks. Our method, with the HuBERT encoder, surpasses the performance of other state-of-the-art architectures, whether trained in supervised or self-supervised settings on the same datasets. Specifically, we achieved F-values of 0.8427 on the Buckeye dataset and 0.7436 on the TIMIT dataset, along with R-values of 0.8489 and 0.7807, respectively. These results establish a new state-of-the-art for both datasets. Beyond the immediate task, our approach offers a robust and efficient preprocessing method for future research in audio tokenization.

The quantification of stenosis severity from X-ray catheter angiography is a challenging task. Indeed, this requires to fully understand the lesion’s geometry by analyzing dynamics of the contrast material, only relying on visual observation by clinicians. To support decision making for cardiac intervention, we propose a hybrid CNN-Transformer model for the assessment of angiography-based non-invasive fractional flow-reserve (FFR) and instantaneous wave-free ratio (iFR) of intermediate coronary stenosis. Our approach predicts whether a coronary artery stenosis is hemodynamically significant and provides direct FFR and iFR estimates. This is achieved through a combination of regression and classification branches that forces the model to focus on the cut-off region of FFR (around 0.8 FFR value), which is highly critical for decision-making. We also propose a spatio-temporal factorization mechanisms that redesigns the transformer’s self-attention mechanism to capture both local spatial and temporal interactions between vessel geometry, blood flow dynamics, and lesion morphology. The proposed method achieves state-of-the-art performance on a dataset of 778 exams from 389 patients. Unlike existing methods, our approach employs a single angiography view and does not require knowledge of the key frame; supervision at training time is provided by a classification loss (based on a threshold of the FFR/iFR values) and a regression loss for direct estimation. Finally, the analysis of model interpretability and calibration shows that, in spite of the complexity of angiographic imaging data, our method can robustly identify the location of the stenosis and correlate prediction uncertainty to the provided output scores.

Terrain traversability estimation is a fundamental task for supporting robot navigation on uneven surfaces. Recent learning-based approaches for predicting traversability from RGB images have shown promising results, but require manual annotation of a large number of images for training. To address this limitation, we present a method for traversability estimation on unlabeled videos that combines dataset synthesis, self-supervision and unsupervised domain adaptation. We pose the traversability estimation as a vector regression task over vertical bands of the observed frame. The model is pre-trained through self-supervision to reduce the distribution shift between synthetic and real data and encourage shared feature learning. Then, supervised training on synthetic videos is carried out, while employing an unsupervised domain adaptation loss to improve its generalization capabilities on real scenes. Experimental results show that our approach is on par with standard supervised training, and effectively supports robot navigation without the need of manual annotations. Training code and synthetic dataset will be publicly released at: https://github.com/perceivelab/traversability-synth .

According to current estimates, patients with cognitive disorders (for example due to stroke, Parkinson's and Alzheimer's disease), dismissed from hospitalization, are subject to a pathological recurrence due to the absence of post-discharge rehabilitation activities; they lose the functional recovery obtained during hospitalization and neurological and cognitive functions often decline. However, telerehabilitation may support effective post-hospitalization treatments by allowing physicians to assign tasks remotely to patients and to monitor their progress, thus reducing overall costs both for national healthcare systems and for patients. Nevertheless, remote and at-home rehabilitation pose several challenges, especially, with the monitoring of the level of engagement of patients during rehabilitation therapy execution. Indeed, while remote rehabilitation has several advantages over standard clinical routine, it is necessary to strictly follow rehabilitation exercises to prevent them from being ineffective. Given these premises, the REHASTART project proposes a platform based on a deep learning framework for monitoring patients' engagement through automated classification of facial expressions and attention levels during exercise execution. More specifically, the proposed approach foresees reliable gaze estimation and emotion recognition through vision transformers. Performance analysis shows that the proposed approach achieves satisfactory accuracy in both facial expression classification and gaze estimation when tested on patients showing motion and cognitive deficits. The results of the deep learning model may be used as a feedback to physicians to monitor training sessions, and to tune them suitably to maximize the effectiveness for each patient.

Research in software vulnerability detection has seen significant growth, with numerous systems and techniques being developed. Deep learning approaches have become partic-ularly popular, with various architectures being adapted for this purpose. Llama 3, the newest AI model from Meta, was released in April 2024. It has been trained on an extensive text corpus and contains four times the amount of code compared to its prede-cessor, Llama 2. In contrast, Code Llama stands out as the only model in the Llama series that has been pre-trained specifically on source code. In this study, we examine the effectiveness of the Llama architectures in static security analysis tasks by fine-tuning Llama 3 and Code Llama for vulnerability classification and detection with high precision. To provide comprehensive insights, we compare their performance against three leading models-CodeBERT, PolyCoder, and NatGen-known for their effectiveness in source code analysis, using two benchmark C/C++ datasets.

Bharadwaj et al. [1] present a comments paper evaluating the classification accuracy of several state-of-the-art methods using EEG data averaged over random class samples. According to the results, some of the methods achieve above-chance accuracy, while the method proposed in [2], that is the target of their analysis, does not. In this rebuttal, we address these claims and explain why they are not grounded in the cognitive neuroscience literature, and why the evaluation procedure is ineffective and unfair.

Objective cognitive assessment is critical for the early detection and management of cognitive decline. The Mini-Mental State Examination (MMSE) is a widely used tool for this purpose, but it requires face-to-face interaction and manual scoring by clinicians. Recent advances in computer vision and deep learning offer the potential to automate and enhance the accuracy of such assessments. This study presents a novel deep learning model that integrates multimodal data captured during cognitive testing sessions on a tablet. By focusing on facial movements, which are captured and magnified through a pre-processing pipeline, the model classifies inputs into categories corresponding to MMSE scores. Our results show a significant correlation between facial movements and MMSE, suggesting the feasibility of using automated video analysis as a reliable proxy for cognitive assessment.

Understanding complex animal behaviors hinges on deciphering the neural activity patterns within brain circuits, making the ability to forecast neural activity crucial for developing predictive models of brain dynamics. This capability holds immense value for neuroscience, particularly in applications such as real-time optogenetic interventions. While traditional encoding and decoding methods have been used to map external variables to neural activity and vice versa, they focus on interpreting past data. In contrast, neural forecasting aims to predict future neural activity, presenting a unique and challenging task due to the spatiotemporal sparsity and complex dependencies of neural signals. Existing transformer-based forecasting methods, while effective in many domains, struggle to capture the distinctiveness of neural signals characterized by spatiotemporal sparsity and intricate dependencies. To address this challenge, we here introduce QuantFormer, a transformer-based model specifically designed for forecasting neural activity from two-photon calcium imaging data. Unlike conventional regression-based approaches, QuantFormerreframes the forecasting task as a classification problem via dynamic signal quantization, enabling more effective learning of sparse neural activation patterns. Additionally, QuantFormer tackles the challenge of analyzing multivariate signals from an arbitrary number of neurons by incorporating neuron-specific tokens, allowing scalability across diverse neuronal populations. Trained with unsupervised quantization on the Allen dataset, QuantFormer sets a new benchmark in forecasting mouse visual cortex activity. It demonstrates robust performance and generalization across various stimuli and individuals, paving the way for a foundational model in neural signal prediction.