Astrometric jitter noise arises when starspots on a rotating stellar surface move in and out of view, shifting the photocenter. This noise may limit our ability to detect and weigh small, sub-Neptune-sized planets around active stars. By deriving a linear forward model for the astrometric jitter signal of a rotating star in a spherical-harmonic coordinate system, we show that jitter noise can be used to reconstruct surface-brightness maps, and in principle, disentangle jitter from stellar reflex motion. Furthermore, we show that astrometry and photometry probe complementary stellar surface information: photometry measures even-degree spherical harmonic surfaces that are symmetric about the equator, while astrometry measures odd-degree modes. Their joint use, therefore, breaks degeneracies in stellar surface mapping. Our model further quantifies the variation in the astrometric signal with inclination angle, which is foundational for studies of worst-case configurations of astrometric starspot noise. For example, we show that pole-on stellar inclinations lead to poorly constrained inversions, as any stellar surface produces a purely circular astrometric jitter signal. We characterize the degeneracy in jointly identifying the stellar surface and stellar inclination, and develop a surface estimation approach. Using this approach, we present example simulations and reconstructions that demonstrate the use of astrometry data alongside light-curve data to improve stellar surface mapping and localize spot positions in latitude and longitude. With forthcoming high-precision Gaia astrometry, astrometric surface mapping provides a promising new approach to probe stellar activity.

Astrometry measures shifts in a star's photocentre and can be used to detect reflex motion due to orbiting exoplanets. Brightness asymmetries (e.g. starspots) rotating in and out of view can also cause apparent motion of the photocenter, termed astrometric jitter, that has previously been considered a source of noise. Here, we explore whether it can be used to map stellar surfaces. We derive a Cramer-Rao bound on the minimum variance for which a stellar surface can theoretically be estimated, quantifying the information content in rotational astrometric jitter. To regularize and break singularities in the Fisher information, we impose a spatial-smoothness Gaussian-Markov random field prior. A key challenge in mapping surfaces arises for stars with unknown rotational axis inclinations, requiring joint estimation of the inclination and the stellar surface. We characterize the coupling between them and quantify the precision gain when inclination is known versus unknown.

Large reasoning models (LRMs), such as OpenAI's o1 and DeepSeek-R1, harness test-time scaling to perform multi-step reasoning for complex problem-solving. This reasoning process, executed before producing final answers, is often guided by special juncture tokens that prompt self-evaluative reflection. We refer to these transition markers and reflective cues as “reflection tokens” (e.g., “wait,” “but,” “alternatively”). In this work, we treat reflection tokens as a “resource” and introduce the problem of resource allocation, aimed at improving the test-time compute performance of LRMs by adaptively regulating the frequency and placement of reflection tokens. Through empirical analysis, we show that both excessive and insufficient use of reflection tokens, referred to as over-reflection and under-reflection, can degrade model performance. To better understand this trade-off, we draw an analogy between reflection token usage and learning rate scheduling in optimization. Building on this insight, we propose cyclical reflection token scheduling (termed Ours), a training-free decoding strategy that dynamically modulates reflection token logits with a bidirectional, position-dependent triangular waveform, incurring no additional computation cost. Experiments on MATH500, AIME2024/2025, AMC2023, GPQA Diamond, and LiveCodeBench demonstrate that Ours consistently improves performance across model sizes (1.5B–8B), outperforming standard decoding and recent approaches such as TIP (thought switching penalty) and S1.

Motivated by the practical application of beam tracking of multiple devices in Multiple Input Multiple Output (MIMO) communication, we study the problem of non-adaptive twenty questions estimation for locating and tracking multiple moving targets under a query-dependent noisy channel. Specifically, we derive a non-asymptotic bound and a second-order asymptotic bound on resolution for optimal query procedures and provide numerical examples to illustrate our results. In particular, we demonstrate that the bound is achieved by a state estimator that thresholds the mutual information density over possible target locations. This single threshold decoding rule has reduced the computational complexity compared to the multiple threshold scheme proposed for locating multiple stationary targets (Zhou, Bai and Hero, TIT 2022). We discuss two special cases of our setting: the case with unknown initial location and known velocity, and the case with known initial location and unknown velocity. Both cases share the same theoretical benchmark that applies to stationary multiple target search in Zhou, Bai and Hero (TIT 2022) while the known initial location case is close to the theoretical benchmark for stationary target search when the maximal speed is inversely proportional to the number of queries. We also generalize our results to account for a piecewise constant velocity model introduced in Zhou and Hero (TIT 2023), where targets change velocity periodically. Finally, we illustrate our proposed algorithm for the application of beam tracking of multiple mobile transmitters in a 5G wireless network.

Inference of community structure in probabilistic graphical models may not be consistent with fairness constraints when nodes have demographic attributes. Certain demographics may be over-represented in some detected communities and under-represented in others. This paper defines a novel $\ell_1$-regularized pseudo-likelihood approach for fair graphical model selection. In particular, we assume there is some community or clustering structure in the true underlying graph, and we seek to learn a sparse undirected graph and its communities from the data such that demographic groups are fairly represented within the communities. In the case when the graph is known a priori, we provide a convex semidefinite programming approach for fair community detection. We establish the statistical consistency of the proposed method for both a Gaussian graphical model and an Ising model for, respectively, continuous and binary data, proving that our method can recover the graphs and their fair communities with high probability.

Self-supervised contrastive learning (SSCL) has emerged as a powerful paradigm for representation learning and has been studied from multiple perspectives, including mutual information and geometric viewpoints. However, supervised contrastive (SupCon) approaches have received comparatively little attention in this context: for instance, while InfoNCE used in SSCL is known to form a lower bound on mutual information (MI), the relationship between SupCon and MI remains unexplored. To address this gap, we introduce ProjNCE, a generalization of the InfoNCE loss that unifies supervised and self-supervised contrastive objectives by incorporating projection functions and an adjustment term for negative pairs. We prove that ProjNCE constitutes a valid MI bound and affords greater flexibility in selecting projection strategies for class embeddings. Building on this flexibility, we further explore the centroid-based class embeddings in SupCon by exploring a variety of projection methods. Extensive experiments on image and audio datasets demonstrate that ProjNCE consistently outperforms both SupCon and standard cross-entropy training. Our work thus refines SupCon along two complementary perspectives–information-theoretic and projection viewpoints–and offers broadly applicable improvements whenever SupCon serves as the foundational contrastive objective.

We propose an alternative to the standard GAN training approach, in which the discriminator is a binary classifier trained by cross-entropy to distinguish real samples from generated ones. Instead, we directly target the discrimination Bayes error rate (BER). To this end, we use the recently proposed Bayes optimal learning threshold (BOLT) loss and train the generator to maximize a surrogate of the discrimination BER. This viewpoint gives a unified perspective on GAN training: different objectives can be interpreted as parameterized bounds on the discrimination BER that describe a trade-off between smoothness and tightness. We show that, under balanced class priors, maximizing the surrogate BER with an unconstrained discriminator minimizes the total variation between the data and generator distributions. By constraining the discriminator to be 1-Lipschitz, the proposed maximization objective defines a discrepancy that is upper-bounded by the Wasserstein-1 distance, thereby linking it to Wasserstein GAN. Experiments on several image-generation datasets under matched architectures and optimization settings show that GAN training using the surrogate BER improves sample quality and coverage over standard baselines. This analysis suggests that the proposed Bayesian viewpoint can achieve a better trade-off between training stability and convergence of the generator to the data distribution.

We consider the problem of the optimal allocation of vaccination and protection measures for the Susceptible-Infected-Recovered-Infected (SIRI) epidemiological model, which generalizes the classical Susceptible-Infected-Recovered (SIR) and Susceptible-Infected-Susceptible (SIS) epidemiological models by allowing for reinfection. First, we introduce the controlled SIRI dynamical model, and discuss the existence and stability of the equilibrium points. Then, we formulate a finite-horizon optimal control problem where the cost of vaccination and protection is proportional to the mass of the population that adopts it. Our main contribution in this work arises from a detailed investigation into the existence/non-existence of singular control inputs, and establishing optimality of bang-bang controls. The optimality of bang-bang control is established by solving an optimal control problem with a running cost that is linear with respect to the input variables. The input variables are associated with actions including the vaccination and imposition of protective measures (e.g., masking or isolation). In contrast to most prior works, we rigorously establish the non-existence of singular controls (i.e., the optimality of bang-bang control for our SIRI model). Under the assumption that the reinfection rate exceeds the first-time infection rate, we characterize the structure of both the optimal control inputs, and establish that the vaccination control input admits a bang-bang structure. The numerical results provide valuable insights into the evolution of the disease spread under optimal control.

Twenty Questions originated as a parlor game between two players. The game starts from a player named an oracle, who privately thinks of a secret. The other player, called the questioner, tries to guess the secret by querying the oracle with at most twenty questions having Yes/No answers. Early versions of the game can be traced to ancient Greece and ancient Rome. Motivated by the Hungarian version of this game, in the middle of the twentieth century, R & eacute;nyi formulated the game as a mathematical problem of guessing an integer from a finite set, where the oracle could lie either randomly to each question or lie to a finite number of questions. The mathematical study of Twenty Questions is motivated by current applications in many domains: communications; ma-chine learning; and computer vision. The game with an oracle who is allowed a fixed number of lies was also studied by Ulam and Berlekamp and is known as the R & eacute;nyi-Ulam-Berlekamp game. In contrast, the setting where the oracle lies randomly is less understood. In this monograph, we summarize recent advances in the information theoretical analysis of Twenty Questions with random error. In particular, focusing on the practical application of sensor network target localization, we study a query-dependent channel to model oracle's noisy response behavior, such as providing a wrong answer or declining to answer a question. We concentrate on non-adaptive query procedures where all questions are designed prior to posing questions. We cover settings relevant to estimating a single target, a single moving target, and multiple targets over the unit cube of a finite dimension. We also consider adaptive querying for a single target to illustrate the benefit of adaptivity. In adaptive querying, each question is designed sequentially using responses to all previous questions. All of our theoretical results are illustrated using numerical examples. Finally, we discuss future research directions. These include geometry constraints for query sets, low-complexity query procedures, connections to group testing, and practical applications in machine learning and communications.

This paper addresses the gradient coding and coded matrix multiplication problems in distributed optimization and coded computing. We present a computationally efficient coding method which overcomes the drawbacks of the Fractional Repetition Coding gradient coding method proposed by Tandon et al., and can also be leveraged by coded computing networks whose servers are of heterogeneous nature. Specifically, we propose a construction for fractional repetition gradient coding; while ensuring that the generator matrix remains close to perfectly balanced for any set of coding parameters, as well as a low complexity decoding step. The proposed binary encoding avoids operations over the real and complex numbers which inherently introduce numerical and rounding errors, thereby enabling accurate distributed encodings of the partial gradients. We then make connections between gradient coding and coded matrix multiplication. Specifically, we show that any gradient coding scheme can be extended to coded matrix multiplication. Furthermore, we show how the proposed binary gradient coding scheme can be used to construct two different coded matrix multiplication schemes, each achieving different trade-offs.

Deterministic embeddings learned by contrastive learning (CL) methods such as SimCLR and SupCon achieve state-of-the-art performance but lack a principled mechanism for uncertainty quantification. We propose Variational Contrastive Learning (VCL), a decoder-free framework that maximizes the evidence lower bound (ELBO) by interpreting the InfoNCE loss as a surrogate reconstruction term and adding a KL divergence regularizer to a uniform prior on the unit hypersphere. We model the approximate posterior q_θ(z|x) as a projected normal distribution, enabling the sampling of probabilistic embeddings. Our two instantiation–VSimCLR and VSupCon–replace deterministic embeddings with samples from q_θ(z|x) and incorporate a normalized KL term into the loss. Experiments on multiple benchmarks demonstrate that VCL mitigates dimensional collapse, enhances mutual information with class labels, and matches or outperforms deterministic baselines in classification accuracy, all the while providing meaningful uncertainty estimates through the posterior model. VCL thus equips contrastive learning with a probabilistic foundation, serving as a new basis for contrastive approaches.

We address the problem of detecting a change in the distribution of a high-dimensional multivariate normal time series. Assuming that the post-change parameters are unknown and estimated using a window of historical data, we extend the framework of quickest change detection (QCD) to the highdimensional setting in which the number of variables increases proportionally with the size of the window used to estimate the post-change parameters. Our analysis reveals that an information theoretic quantity, which we call the Normalized High- Dimensional Kullback-Leibler divergence (NHDKL), governs the high-dimensional asymptotic performance of QCD procedures. Specifically, we show that the detection delay is asymptotically inversely proportional to the difference between the NHDKL of the true post-change versus pre-change distributions and the NHDKL of the true versus estimated post-change distributions. In cases of perfect estimation, where the latter NHDKL is zero, the delay is inversely proportional to the NHDKL between the post-change and pre-change distributions alone. Thus, our analysis is a direct generalization of the traditional fixed-dimension, large-sample asymptotic framework, where the standard KL divergence is asymptotically inversely proportional to detection delay. Finally, we identify parameter estimators that asymptotically minimize the NHDKL between the true versus estimated post-change distributions, resulting in a QCD method that is guaranteed to outperform standard approaches based on fixed-dimension asymptotics.

This paper proposes a hierarchical Bayesian multitask learning model that is applicable to the general multi-task binary classification learning problem where the model assumes a shared sparsity structure across different tasks. We derive a computationally efficient inference algorithm based on variational inference to approximate the posterior distribution. We demonstrate the potential of the new approach on various synthetic datasets and for predicting human health status based on microbiome profile. Our analysis incorporates data pooled from multiple microbiome studies, along with a comprehensive comparison with other benchmark methods. Results in synthetic datasets show that the proposed approach has superior support recovery property when the underlying regression coefficients share a common sparsity structure across different tasks. Our experiments on microbiome classification demonstrate the utility of the method in extracting informative taxa while providing well-calibrated predictions with uncertainty quantification and achieving competitive performance in terms of prediction metrics. Notably, despite the heterogeneity of the pooled datasets (e.g., different experimental objectives, laboratory setups, sequencing equipment, patient demographics), our method delivers robust results.

The trustworthy machine learning (ML) community is increasingly recognizing the crucial need for models capable of selectively ‘unlearning’ data points after training. This leads to the problem of machine unlearning (MU), aiming to eliminate the influence of chosen data points on model performance, while still maintaining the model’s utility post-unlearning. Despite various MU methods for data influence erasure, evaluations have largely focused on random data forgetting, ignoring the vital inquiry into which subset should be chosen to truly gauge the authenticity of unlearning performance. To tackle this issue, we introduce a new evaluative angle for MU from an adversarial viewpoint. We propose identifying the data subset that presents the most significant challenge for influence erasure, i.e., pinpointing the worst-case forget set. Utilizing a bi-level optimization principle, we amplify unlearning challenges at the upper optimization level to emulate worst-case scenarios, while simultaneously engaging in standard training and unlearning at the lower level, achieving a balance between data influence erasure and model utility. Our proposal offers a worst-case evaluation of MU’s resilience and effectiveness. Through extensive experiments across different datasets (including CIFAR-10, 100, CelebA, Tiny ImageNet, and ImageNet) and models (including both image classifiers and generative models), we expose critical pros and cons in existing (approximate) unlearning strategies. Our results illuminate the complex challenges of MU in practice, guiding the future development of more accurate and robust unlearning algorithms. The code and supplementary material (appendix) are available at https://github.com/OPTML-Group/Unlearn-WorstCase .

Disease risk prediction models play an important role in preventing disease developments in modern healthcare. However, the lack of focus on high-risk patients has hindered the large-scale practical application of these models, especially considering the limitation of medical resources available for following up on patients who are deemed high-risk. In this study, we propose a novel and practical approach that focuses on minimizing the number of false positive observations among high-risk patients by introducing the Highest-k Loss. The solution is to estimate the weights of the highest k scores with a differentiable estimation of the sorting operation and apply the weights to the loss function. We extracted 253,680 survey responses from a public dataset of the U.S. health survey system to define a diabetes prediction task. This study employs nested cross-validation as well as an aggregated model applied to an independent test set to systematically evaluate the proposed method. Compared with traditional binary cross entropy loss and Focal loss, the Highest-k loss improved the precision (positive predictive value) for the highest 1% scores by 0.05 (95% CI: 0.041-0.055), the highest 5% scores by 0.03 (95% CI: 0.024-0.032), and the highest 10% scores by 0.02 (95% CI: 0.016-0.021). The introduced Highest-k loss function addresses the problem of prevailing risk prediction models and offers a practical solution that focuses on patients with the k highest predictive scores who can realistically receive an intervention as opposed to the entire patient population.

The behavior and chemical reactivity of a group of molecules are typically illustrated using intricate and extensive chemical reaction networks. These networks consist of a group of species and a series of reactions that detail their evolution. Although recent years have seen a surge in efforts to create numerical algorithms that generate dense chemical reaction networks with thousands of reactions and molecules, the simulation of these networks is a computationally demanding task, even for reaction mechanisms that describe the combustion of hydrocarbons. This paper introduces an innovative and unbiased approach to data-driven model reduction of extensive reaction networks called the SParse IdeNtification (SPIN) algorithm. SPIN combines tools from different domains to identify a set of crucial reactions using species concentrations and reaction rates, all while maintaining minimal computational costs and without requiring extra data or simulations. SPIN is successfully tested for large combustion networks of propane and n-heptane. The study demonstrates that, despite containing only one-fifth of the reactions found in the full mechanism, the SPIN reduced mechanism for n-heptane combustion serves as a highly accurate approximation of the original mechanism, with an average deviation of only 8.4% in ignition delay. Notably, this outstanding performance is achieved without bias towards any particular target property, such as ignition delay, as the reduction and model parameters are optimized to obtain the best possible results. We demonstrate that SPIN can operate as a standalone method or be hybridized with existing species-based reduction methods to further enhance its ability to identify the most significant reactions. This capability is particularly beneficial in comprehending the intricate mechanisms of combustion.

Linear regression is a fundamental and primitive problem in supervised machine learning, with applications ranging from epidemiology to finance. In this work, we propose methods for speeding up distributed linear regression. We do so by leveraging randomized techniques, while also ensuring security and straggler resiliency in asynchronous distributed computing systems. Specifically, we randomly rotate the basis of the system of equations and then subsample blocks, to simultaneously secure the information and reduce the dimension of the regression problem. In our setup, the basis rotation corresponds to an encoded encryption in an approximate gradient coding scheme, and the subsampling corresponds to the responses of the non-straggling servers in the centralized coded computing framework. This results in a distributive iterative stochastic approach for matrix compression and steepest descent.

A unified representation space in multi-modal learning is essential for effectively integrating diverse data sources, such as text, images, and audio, to enhance efficiency and performance across various downstream tasks. Recent binding methods, such as ImageBind, typically rely on a single, fixed anchor modality for aligning multi-modal data. We mathematically analyze these fixed anchor binding methods and uncover significant limitations: (1) over-reliance on the choice of the anchor modality, (2) inadequate capture of intra-modal information, and (3) failure to account for cross-modal correlation among non-anchored modalities. To address these issues, we propose the need for adaptive anchor binding methods, exemplified by our framework CentroBind. The proposed method uses adaptively adjustable centroid-based anchors generated from all available modalities, leading to a balanced and rich representation space. We theoretically demonstrate that our approach captures three critical properties of multi-modal learning – intra-modal learning, inter-modal learning, and multi-modal alignment – while constructing a unified representation that spans all modalities. Experiments on both synthetic and real-world datasets show that adaptive anchor methods such as CentroBind consistently outperform fixed anchor binding methods, verifying our analysis.

We revisit the problem of statistical sequence matching between two databases of sequences initiated by Unnikrishnan (TIT 2015) and derive theoretical performance guarantees for the generalized likelihood ratio test (GLRT). We first consider the case where the number of matched pairs of sequences between the databases is known. In this case, the task is to accurately find the matched pairs of sequences among all possible matches between the sequences in the two databases. We analyze the performance of the GLRT by Unnikrishnan and explicitly characterize the tradeoff between the mismatch and false reject probabilities under each hypothesis in both large and small deviations regimes. Furthermore, we demonstrate the optimality of Unnikrishnan’s GLRT test under the generalized Neyman-Person criterion for both regimes and illustrate our theoretical results via numerical examples. Subsequently, we generalize our achievability analyses to the case where the number of matched pairs is unknown, and an additional error probability needs to be considered. When one of the two databases contains a single sequence, the problem of statistical sequence matching specializes to the problem of multiple classification introduced by Gutman (TIT 1989). For this special case, our result for the small deviations regime strengthens previous result of Zhou, Tan and Motani (Information and Inference 2020) by removing unnecessary conditions on the generating distributions.

Deep learning models have shown considerable vulnerability to adversarial attacks, particularly as attacker strategies become more sophisticated. While traditional adversarial training (AT) techniques offer some resilience, they often focus on defending against a single type of attack, e.g., the l(infinity)-norm attack, which can fail for other types. This paper introduces a computationally efficient multilevel l(p) defense, called the Efficient Robust Mode Connectivity (EMRC) method, which aims to enhance a deep learning model's resilience against multiple l(p)-norm attacks. Similar to analytical continuation approaches used in continuous optimization, the method blends two p-specific adversarially optimal models, the l(1)- and l(infinity)-norm AT solutions, to provide good adversarial robustness for a range of p. We present experiments demonstrating that our approach performs better on various attacks as compared to AT-l(infinity), E-AT, and MSD, for datasets/architectures including: CIFAR-10, CIFAR-100 / PreResNet110, WideResNet, ViT-Base.