Large Multimodal Models (LMMs) have recently emerged as promising backbones for GUI-agent models, where high-resolution GUI screenshots are introduced to the prompts at each iteration step. However, these screenshots exhibit highly non-uniform spatial information density: large regions may carry little information and are visually homogeneous, while key text and icons may require high visual fidelity. Existing approaches to this problem either require additional training or rely on attention-based token compression, ignoring the structured layout and spatial redundancy of GUI screenshots. To fill the gap, this paper proposes AquaUI, a training-free inference-time token reduction method for GUI agent models that utilizes the non-uniform information density in screenshots. AQuaUI constructs an adaptive quadtree on each screenshot input and keeps one representative merged token per leaf of the quadtree. AQuaUI preserves the spatial positions of retained tokens throughout the pipeline to ensure that all position-encoding stages remain consistent. To further improve temporal consistency across multi-step GUI interactions, we propose a conditional quadtree algorithm that leverages the continuity between consecutive screenshots within a single request. Specifically, it refines the current quadtree using previous quadtrees as references, helping preserve fine-grained regions across static or mildly shifted GUI states. We implement AQuaUI on state-of-the-art GUI agent models and conduct experiments on standard grounding and navigational benchmarks. AQuaUI consistently shows improved accuracy-efficiency trade-offs over prior baselines. Notably, on GUI-Owl-1.5-32B-Instruct, AQuaUI achieves up to 13.22

Evaluating domain generalization (DG) for foundational models like CLIP is challenging, as web-scale pretraining data potentially covers many existing benchmarks. Consequently, current DG evaluation may neither be sufficiently challenging nor adequately test genuinely unseen data scenarios. To better assess the performance of CLIP on DG in-the-wild, a scenario where CLIP encounters challenging unseen data, we consider two approaches: (1) evaluating on 33 diverse datasets with quantified out-of-distribution (OOD) scores after fine-tuning CLIP on ImageNet, and (2) using unlearning to make CLIP `forget' some domains as an approximation. We observe that CLIP's performance deteriorates significantly on more OOD datasets. To address this, we present CLIP-DCA (Disentangling Classification from enhanced domain Aware representations). Our approach is motivated by the observation that while standard domain invariance losses aim to make representations domain-invariant, this can be harmful to foundation models by forcing the discarding of domain-aware representations beneficial for generalization. We instead hypothesize that enhancing domain awareness is a prerequisite for effective domain-invariant classification in foundation models. CLIP-DCA identifies and enhances domain awareness within CLIP's encoders using a separate domain head and synthetically generated diverse domain data. Simultaneously, it encourages domain-invariant classification through disentanglement from the domain features. CLIP-DCA shows significant improvements within this challenging evaluation compared to existing methods, particularly on datasets that are more OOD.

Semi-supervised domain generalization (SSDG) aims to solve the problem of generalizing to out-of-distribution data when only a few labels are available. Due to label scarcity, applying domain generalization methods often underperform. Consequently, existing SSDG methods combine semi-supervised learning methods with various regularization terms. However, these methods do not explicitly regularize to learn domains invariant representations across all domains, which is a key goal for domain generalization. To address this, we introduce FixCLR. Inspired by success in self-supervised learning, we change two crucial components to adapt contrastive learning for explicit domain invariance regularization: utilization of class information from pseudo-labels and using only a repelling term. FixCLR can also be added on top of most existing SSDG and semi-supervised methods for complementary performance improvements. Our research includes extensive experiments that have not been previously explored in SSDG studies. These experiments include benchmarking different improvements to semi-supervised methods, evaluating the performance of pretrained versus non-pretrained models, and testing on datasets with many domains. Overall, FixCLR proves to be an effective SSDG method, especially when combined with other semi-supervised methods.

In survival analysis, subjects often face competing risks; for example, individuals with cancer may also suffer from heart disease or other illnesses, which can jointly influence the prognosis of risks and censoring. Traditional survival analysis methods often treat competing risks as independent and fail to accommodate the dependencies between different conditions. In this paper, we introduce HACSurv, a survival analysis method that learns Hierarchical Archimedean Copulas structures and cause-specific survival functions from data with competing risks. HACSurv employs a flexible dependency structure using hierarchical Archimedean copulas to represent the relationships between competing risks and censoring. By capturing the dependencies between risks and censoring, HACSurv improves the accuracy of survival predictions and offers insights into risk interactions. Experiments on synthetic dataset demonstrate that our method can accurately identify the complex dependency structure and precisely predict survival distributions, whereas the compared methods exhibit significant deviations between their predictions and the true distributions. Experiments on multiple real-world datasets also demonstrate that our method achieves better survival prediction compared to previous state-of-the-art methods.

The early detection of Alzheimer’s disease (AD) is important for effective therapeutic interventions and optimized enrollment for clinical trials. Recent studies have shown high accuracy in identifying mild AD by applying visibility graph and machine learning methods to electroencephalographic (EEG) data. We present a novel analytical framework combining Weighted Visibility Graphs (WVG) and ensemble learning to detect individuals in the “preclinical” stage of AD (preAD) using a word repetition EEG paradigm, where WVG is an advanced variant of natural Visibility Graph (VG), incorporating weighted edges based on the visibility degree between corresponding data points. The EEG signals were recorded from 40 cognitively unimpaired elderly participants (20 preclinical AD and 20 normal old) during a word repetition task. Event-related potential (ERP) and oscillatory signals were extracted from each EEG channel and transformed into a WVG network, from which relevant topological features were extracted. The features were selected using t-tests to reduce noise. Subsequent statistical analysis reveals significant disparities in the structure of WVG networks between preAD and normal subjects. Furthermore, Principal Component Analysis (PCA) was applied to condense the input data into its principal features. Leveraging these PCA components as input features, several machine learning algorithms are used to classify preAD vs. normal subjects. To enhance classification accuracy and robustness, an ensemble method is employed alongside the classifiers. Our framework achieved an accuracy of up to 92% discriminating preAD from normal old using both linear and non-linear classifiers, signifying the efficacy of combining WVG and ensemble learning in identifying very early AD from EEG signals. The framework can also improve clinical efficiency by reducing the amount of data required for effective classification and thus saving valuable clinical time.

A prevailing approach to explain time series models is to generate attribution in time domain input. A recent development in time series XAI is the concept of explanation spaces, where any model trained in the time domain can be interpreted with any existing XAI method in alternative domains, such as frequency or time-frequency domain. The prevailing approach is to present XAI attributions either in the time domain or in the domain where the attribution is most sparse. In this paper, we demonstrate that in certain cases, XAI methods can generate attributions that highlight fundamentally different features in the time and frequency domains that are not direct counterparts of one another. This observation suggests that both domains' attributions should be presented to achieve a more comprehensive interpretation. Thus it shows the necessity of multi-domain explanation. To quantify when such cases arise, we introduce the uncertainty principle (UP), originally developed in quantum mechanics and later studied in harmonic analysis and signal processing, to the XAI literature. This principle establishes a lower bound on how much a signal can be simultaneously localized in both the time and frequency domains. By leveraging this concept, we assess whether attributions in the time and frequency domains violate this bound, indicating that they emphasize distinct features. In other words, UP provides a sufficient condition that the time and frequency domain explanations do not match and, hence, should be both presented to the end user. We validate the effectiveness of this approach across various deep learning models, XAI methods, and a wide range of classification and forecasting datasets. The frequent occurrence of UP violations across various datasets and XAI methods highlights the limitations of existing approaches that focus solely on time-domain explanations. This underscores the need for multi-domain explanations as a new paradigm. The source code is available at https://github.com/shrezaei/TSX-spaces

Explainability in time series models is crucial for fostering trust, facilitating debugging, and ensuring interpretability in real-world applications. In this work, we introduce Implet, a novel post-hoc explainer that generates accurate and concise subsequence-level explanations for time series models. Our approach identifies critical temporal segments that significantly contribute to the model's predictions, providing enhanced interpretability beyond traditional feature-attribution methods. Based on it, we propose a cohort-based (group-level) explanation framework designed to further improve the conciseness and interpretability of our explanations. We evaluate Implet on several standard time-series classification benchmarks, demonstrating its effectiveness in improving interpretability. The code is available at https://github.com/LbzSteven/implet

Human understandable explanation of deep learning models is essential for various critical and sensitive applications. Unlike image or tabular data where the importance of each input feature (for the classifier's decision) can be directly projected into the input, time series distinguishable features (e.g. dominant frequency) are often hard to manifest in time domain for a user to easily understand. Additionally, most explanation methods require a baseline value as an indication of the absence of any feature. However, the notion of lack of feature, which is often defined as black pixels for vision tasks or zero/mean values for tabular data, is not well-defined in time series. Despite the adoption of explainable AI methods (XAI) from tabular and vision domain into time series domain, these differences limit the application of these XAI methods in practice. In this paper, we propose a simple yet effective method that allows a model originally trained on the time domain to be interpreted in other explanation spaces using existing methods. We suggest five explanation spaces, each of which can potentially alleviate these issues in certain types of time series. Our method can be easily integrated into existing platforms without any changes to trained models or XAI methods. The source code is available at https://github.com/shrezaei/TS-X-spaces.

Synthetic data has been actively used for various machine learning-based tasks due to its benefits such as massive reproducibility and privacy enhancement compared to using the original data. The quality of the generated synthetic dataset crucially depends on the quality of the original data, and the latter is often corrupted by label noise. While there have been studies on feature noise, how label noise affects synthetic data generation is under-explored. In this paper, we evaluate the impact of the noisy label on synthetic data generation with a focus on tabular data. One challenge is how to evaluate the quality of synthetic data under label noise. To this end, we design comprehensive experiments to measure the impact of label noise on synthetic data generation in different aspects: synthetic data quality, data utility, and convergence for training synthesizers and machine learning models for downstream tasks. The empirical results cover wide aspects of synthetic data generation under label noise and they show quality and utility degrades with higher noise levels while there is no significant effect on the synthesizer convergence observed.

INTRODUCTION:Prediction of amnestic mild cognitive impairment (aMCI) and Alzheimer's disease (AD) using machine learning has primarily focused on short-term predictions spanning 1-3 years. This study aimed to develop a new machine learning technique to extend predictions of cognitive status over 3-10 years from their last visit. METHODS:We leveraged deep learning to analyze two longitudinal feature sets: (1) neuropsychological data and (2) neuropsychological data with the addition of patient history data. We also introduce two modeling techniques: (1) to separate normalized baseline features and deviations from baseline, and (2) a new linear attention-based imputation method. RESULTS:We demonstrate (1) our technique achieves high 1vA accuracy, representing 81.65% for Control, 72.87% for aMCI, and 86.52% for AD on a 3- to 10-year horizon, and (2) the new method is more accurate than previously proposed approaches for this time horizon. DISCUSSION:This work offers a new set of techniques for big-data analysis of longitudinal dementia data. HIGHLIGHTS:Develops a new method for the prediction using deep learning of longitudinally verified amnestic mild cognitive impairment (aMCI) and Alzheimer's disease (AD) using the National Alzheimer's Coordinating Center NACC) database. Demonstrates comparable performance on the 3- to 10-year prediction horizon, which is significantly more challenging to predict than using the previous approach that only used a 1- to 3-year prediction horizon. Highlights that even the prediction of verified 3- to 10-year aMCI that eventually leads to AD is still a challenging task.

Machine learning has been successful in building control policies to drive a complex system to desired states in various applications (e.g. games, robotics, etc.). To be specific, a number of parameters of policy can be automatically optimized from the observations of environment to be able to generate a sequence of decisions leading to the best performance. In this survey paper, we particularly explore such policy-learning techniques for another unique, practical use-case scenario–farming, in which critical decisions (e.g., water supply, heating, etc.) must be made in a timely manner to minimize risks (e.g., damage to plants) while maximizing the revenue (e.g., healthy crops) in the end. We first provide a broad overview of latest studies on it to identify not only domain-specific challenges but opportunities with potential solutions, some of which are suggested as promising directions for future research. Also, we then introduce our successful approach to being ranked second among 46 teams at the ”3rd Autonomous Greenhouse Challenge” to use this specific example to discuss the lessons learned about important considerations for design to create autonomous farm-management systems.

Recent developments in deep reinforcement learning have been very successful in learning complex, previously intractable problems. Sample efficiency and local optimality, however, remain significant challenges. To address these challenges, novelty-driven exploration strategies have emerged and shown promising potential. Unfortunately, no single algorithm outperforms all others in all tasks and most of them struggle with tasks with high-dimensional and complex observations. In this work, we propose Adventurer, a novelty-driven exploration algorithm that is based on Bidirectional Generative Adversarial Networks (BiGAN), where BiGAN is trained to estimate state novelty. Intuitively, a generator that has been trained on the distribution of visited states should only be able to generate a state coming from the distribution of visited states. As a result, novel states using the generator to reconstruct input states from certain latent representations would lead to larger reconstruction errors. We show that BiGAN performs well in estimating state novelty for complex observations. This novelty estimation method can be combined with intrinsic-reward-based exploration. Our empirical results show that Adventurer produces competitive results on a range of popular benchmark tasks, including continuous robotic manipulation tasks (e.g. Mujoco robotics) and high-dimensional image-based tasks (e.g. Atari games).

We present a novel visual instruction tuning strategy to improve the zero-shot task generalization of multimodal large language models by building a firm text-only knowledge base. Existing work lacks sufficient experimentation on the importance of each modality in the instruction tuning stage, often using a majority of vision-language data while keeping text-only data limited and fixing mixtures of modalities. By incorporating diverse text-only data in the visual instruction tuning stage, we vary vision-language data in various controlled experiments to investigate the importance of modality in visual instruction tuning. Our comprehensive evaluation shows that the text-heavy instruction tuning approach is able to perform on-par with traditional vision-heavy mixtures on both modalities across 12 general datasets while using as low as half the total training tokens. We find that simply increasing sufficiently diverse text-only data enables transfer of instruction following ability and domain knowledge across modalities while being more efficient than the vision-language approach.

Model-based reinforcement learning (MBRL) has demonstrated superior sample efficiency compared to model-free reinforcement learning (MFRL). However, the presence of inaccurate models can introduce biases during policy learning, resulting in misleading trajectories. The challenge lies in obtaining accurate models due to limited diverse training data, particularly in regions with limited visits (uncertain regions). Existing approaches passively quantify uncertainty after sample generation, failing to actively collect uncertain samples that could enhance state coverage and improve model accuracy. Moreover, MBRL often faces difficulties in making accurate multi-step predictions, thereby impacting overall performance. To address these limitations, we propose a novel framework for uncertainty-aware policy optimization with model-based exploratory planning. In the model-based planning phase, we introduce an uncertainty-aware k-step lookahead planning approach to guide action selection at each step. This process involves a trade-off analysis between model uncertainty and value function approximation error, effectively enhancing policy performance. In the policy optimization phase, we leverage an uncertainty-driven exploratory policy to actively collect diverse training samples, resulting in improved model accuracy and overall performance of the RL agent. Our approach offers flexibility and applicability to tasks with varying state/action spaces and reward structures. We validate its effectiveness through experiments on challenging robotic manipulation tasks and Atari games, surpassing state-of-the-art methods with fewer interactions, thereby leading to significant performance improvements.

Federated learning (FL) is decentralized machine learning framework that finds various applications in health, finance, and the Internet of things. This article studies the under-explored business competition in FL, where organizations are both collaborators in training a shared model and competitors in providing model-based services to a continuum of customers. We focus on an oligopoly case with three organizations. To understand how competition affects FL collaboration, we start with a benchmark case where organizations are not competitors, and show that they have an incentive to collaborate. However, in the presence of competition, organizations may prefer to train local models instead of collaborating via FL (even if FL incurs zero training costs). The reason is that FL intensifies price competition by improving organizations' model performance to a similar level. To address this issue, we devise a model differentiation mechanism in which organizations adaptively adjust their model performance, enabling differentiated model-based services to customers. We prove that the adaptive mechanism converges in polynomial time and is incentive compatible. Perhaps surprisingly, numerical experiments on CIFAR-10 show that the mechanism can simultaneously improve the model performance, organizations' revenues, and social welfare. The improvement is up to 22.31%, 14.42%, and 19.50%, respectively.

INTRODUCTION:The incidence of Alzheimer's disease (AD) and obesity rise concomitantly. This study examined whether factors affecting metabolism, race/ethnicity, and sex are associated with AD development. METHODS:The analyses included patients ≥ 65 years with AD diagnosis in six University of California hospitals between January 2012 and October 2023. The controls were race/ethnicity, sex, and age matched without dementia. Data analyses used the Cox proportional hazards model and machine learning (ML). RESULTS:Hispanic/Latino and Native Hawaiian/Pacific Islander, but not Black subjects, had increased AD risk compared to White subjects. Non-infectious hepatitis and alcohol abuse were significant hazards, and alcohol abuse had a greater impact on women than men. While underweight increased AD risk, overweight or obesity reduced risk. ML confirmed the importance of metabolic laboratory tests in predicting AD development. DISCUSSION:The data stress the significance of metabolism in AD development and the need for racial/ethnic- and sex-specific preventive strategies. HIGHLIGHTS:Hispanics/Latinos and Native Hawaiians/Pacific Islanders show increased hazards of Alzheimer's disease (AD) compared to White subjects. Underweight individuals demonstrate a significantly higher hazard ratio for AD compared to those with normal body mass index. The association between obesity and AD hazard differs among racial groups, with elderly Asian subjects showing increased risk compared to White subjects. Alcohol consumption and non-infectious hepatitis are significant hazards for AD. Machine learning approaches highlight the potential of metabolic panels for AD prediction.

IntroductionAttention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder where hyperactivity often manifests as fidgeting, a non-goal-directed motoric action. Many studies demonstrate fidgeting varies under different conditions as a self-regulating mechanism for attention and alertness during cognitively demanding tasks. Fidgeting has also been associated with reaction time variability. However, a lack of standard variables to define and quantify fidgeting can lead to discrepancies in data and interpretability issues across studies. Furthermore, little is known about fidgeting in adults with ADHD compared to youth. This study aims to design a framework to quantify meaningful fidgeting variables and to apply them to test the relation between fidgeting and performance on a cognitive task, the Flanker, in adults with ADHD.MethodOur study included 70 adult participants diagnosed with ADHD, aged 18–50 years (30.5 ± 7.2 years). Screening included a structured clinical interview, childhood, current self and current observer ratings of ADHD symptoms. Actigraphy devices were attached to the left wrist and right ankle during completion of a cognitive control, attention task (the Flanker). Laboratory testing was subsequently completed on a single day. The relation between task performance, reaction time variability and fidgeting was examined.Results and DiscussionOur analysis revealed increased fidgeting during correct trials as defined by our new variables, consistent with previous observations. Furthermore, differences in fidgeting were observed between early and later trials while the percentage of correct trials were not significantly different. This suggests a relation between the role of fidgeting and sustaining attention. Participants with low reaction time variability, that is, those with more consistent reaction times, fidgeted more during later trials. This observation supports the theory that fidgeting aids arousal and improves sustained attention. Finally, a correlation analysis using ADHD-symptom rating scales validated the relevance of the fidget variables in relation to ADHD symptom severity. These findings suggest fidgeting may be a compensatory mechanism that aids in sustained attention for those with ADHD, although alternative explanations exist.ConclusionOur study suggests that fidgeting may aid in sustained attention during the attention-demanding, cognitive control processes for adults with ADHD, with more fidgeting observed during correct trials and among participants with lower reaction time variability. Furthermore, the newly defined fidget variables were validated through a significant correlation with ADHD rating scales. By sharing our implementation of fidget variables, we hope to standardize and encourage further quantitative research into the role of fidgeting in ADHD.

In cross-silo federated learning (FL), companies collaboratively train a shared global model without sharing heterogeneous data. Prior related work focused on algorithm development to tackle data heterogeneity. However, the dual problem of coopetition, i.e., FL collaboration and market competition, remains under-explored. This paper studies the FL coopetition using a dynamic two-period game model. In period 1, an incumbent company trains a local model and provides model-based services at a chosen price to users. In period 2, an entrant company enters, and both companies decide whether to engage in FL collaboration and then compete in selling model-based services at different prices to users. Analyzing the two-period game is challenging due to data heterogeneity, and that the incumbent's period one pricing has a temporal impact on coopetition in period 2, resulting in a non-concave problem. To address this issue, we decompose the problem into several concave sub-problems and develop an algorithm that achieves a global optimum. Numerical results on three public datasets show two interesting insights. First, FL training brings model performance gain as well as competition loss, and collaboration occurs only when the performance gain outweighs the loss. Second, data heterogeneity can incentivize the incumbent to limit market penetration in period 1 and promote price competition in period 2.