This paper considers the problem of understanding the behavior of a general class of accelerated gradient methods on smooth nonconvex functions. Motivated by some recent works that have proposed effective algorithms, based on Polyak’s heavy ball method and the Nesterov accelerated gradient method, to achieve convergence to a local minimum of nonconvex functions, this work proposes a broad class of Nesterov-type accelerated methods and puts forth a rigorous study of these methods encompassing the escape from saddle points and convergence to local minima through both an asymptotic and a non-asymptotic analysis. In the asymptotic regime, this paper answers an open question of whether Nesterov’s accelerated gradient method (NAG) with variable momentum parameter avoids strict saddle points almost surely. This work also develops two metrics of asymptotic rates of convergence and divergence, and evaluates these two metrics for several popular standard accelerated methods such as the NAG and Nesterov’s accelerated gradient with constant momentum (NCM) near strict saddle points. In the non-asymptotic regime, this work provides an analysis that leads to the “linear” exit time estimates from strict saddle neighborhoods for trajectories of these accelerated methods as well the necessary conditions for the existence of such trajectories. Finally, this work studies a sub-class of accelerated methods that can converge in convex neighborhoods of nonconvex functions with a near optimal rate to a local minimum and at the same time this sub-class offers superior saddle-escape behavior compared to that of NAG.

Empirical risk minimization (ERM) is a cornerstone of modern machine learning (ML), supported by advances in optimization theory that ensure efficient solutions with provable algorithmic and statistical learning rates. Privacy, memory, computation, and communication constraints necessitate data collection, processing, and storage across network-connected devices. In many applications, networks operate in decentralized settings where a central server cannot be assumed, requiring decentralized ML algorithms that are efficient and resilient. Decentralized learning, however, faces significant challenges, including an increased attack surface. This paper focuses on the man-in-the-middle (MITM) attack, wherein adversaries exploit communication vulnerabilities to inject malicious updates during training, potentially causing models to deviate from their intended ERM solutions. To address this challenge, we propose RESIST (Resilient dEcentralized learning using conSensus gradIent deScenT), an optimization algorithm designed to be robust against adversarially compromised communication links, where transmitted information may be arbitrarily altered before being received. Unlike existing adversarially robust decentralized learning methods, which often (i) guarantee convergence only to a neighborhood of the solution, (ii) lack guarantees of linear convergence for strongly convex problems, or (iii) fail to ensure statistical consistency as sample sizes grow, RESIST overcomes all three limitations. It achieves algorithmic and statistical convergence for strongly convex, Polyak-Lojasiewicz, and nonconvex ERM problems by employing a multistep consensus gradient descent framework and robust statistics-based screening methods to mitigate the impact of MITM attacks. Experimental results demonstrate the robustness and scalability of RESIST across attack strategies, screening methods, and loss functions.

The focus of this paper is to improve simulation-based inference through improved denoising of experimental data. Statistical inference of parameters governing a complex physical process from observed data is an important task for several scientific domains. In a likelihood-free inference setup, the inference engine used to make inference from the experimental data can be trained using simulated data. In many scenarios, experimental data is corrupted by noise during the data acquisition process, which is either unaccounted for in the simulator or whose strength might not match with the one set in the simulator. This deteriorates the accuracy of the inference. While advances in denoising can be leveraged to address this challenge, many denoising techniques ignore the fact that experimental data in many scientific domains tends to be irregular/graph-structured. This paper addresses these two challenges by developing a kernel-based learnable graph (KBLG) denoiser, which can be used to denoise experimental graph-structured data. In order to exhibit the efficacy of the developed denoiser in simulation-driven inference problems, this paper considers the problem of inference of the position of an interaction in an astroparticle detector. The available simulated data in this case is the snapshots of the luminous responses of the photomultiplier tube sensors used within the dark matter detection experiment. In experimental situations, these measurements are corrupted by noise generated by secondary optical and electronic processes. The proposed KBLG denoiser is used to denoise the multiple snapshots of the experimental measurements, which are then used for position reconstruction using a multilayer perceptron trained using noiseless simulated data. Numerical results exhibit that the proposed KBLG denoiser outperforms a graph-agnostic denoiser in terms of MLP-based position reconstruction performances for different levels of noise (noise variance).

Recent works have shown that imposing tensor structures on the coefficient tensor in regression problems can lead to more reliable parameter estimation and lower sample complexity compared to vector-based methods. This work investigates a new low-rank tensor model, called Low Separation Rank (LSR), in Generalized Linear Model (GLM) problems. The LSR model -- which generalizes the well-known Tucker and CANDECOMP/PARAFAC (CP) models, and is a special case of the Block Tensor Decomposition (BTD) model -- is imposed onto the coefficient tensor in the GLM model. This work proposes a block coordinate descent algorithm for parameter estimation in LSR-structured tensor GLMs. Most importantly, it derives a minimax lower bound on the error threshold on estimating the coefficient tensor in LSR tensor GLM problems. The minimax bound is proportional to the intrinsic degrees of freedom in the LSR tensor GLM problem, suggesting that its sample complexity may be significantly lower than that of vectorized GLMs. This result can also be specialised to lower bound the estimation error in CP and Tucker-structured GLMs. The derived bounds are comparable to tight bounds in the literature for Tucker linear regression, and the tightness of the minimax lower bound is further assessed numerically. Finally, numerical experiments on synthetic datasets demonstrate the efficacy of the proposed LSR tensor model for three regression types (linear, logistic and Poisson). Experiments on a collection of medical imaging datasets demonstrate the usefulness of the LSR model over other tensor models (Tucker and CP) on real, imbalanced data with limited available samples.

Biomedical imaging systems often produce multidimensional signals (tensors). The high expense of image acquisition limits sample sizes and privacy regulations can prevent centralizing data from multiple sites. Federated learning can allow researchers to form research consortia to perform joint analyses without centralizing data. Standard analysis approaches for tensors often vectorize the data, resulting in high dimensional models for which the total sample size across a consortium may be insufficient. We propose a federated algorithm for tensor regression using generalized linear models (GLMs) based on a recently proposed centralized method using low separation rank (LSR) tensor decompositions. Our results show that by balancing the ratio of local update steps to rounds of communication, we can achieve results similar to those of a centralized algorithm using the entire data.

Federated learning is a technique that allows multiple entities to collaboratively train models using their data without compromising data privacy. However, despite its advantages, federated learning can be susceptible to false data injection attacks. In these scenarios, a malicious entity with control over specific agents in the network can manipulate the learning process, leading to a suboptimal model. Consequently, addressing these data injection attacks presents a significant research challenge in federated learning systems. In this paper, we propose a novel approach to detect and mitigate data injection attacks on federated learning systems. Our mitigation strategy is a local scheme, performed during a single instance of training by the coordinating node, allowing for mitigation during the convergence of the algorithm. Whenever an agent is suspected of being an attacker, its data will be ignored for a certain period; this decision will often be re-evaluated. We prove that with probability one, after a finite time, all attackers will be ignored while the probability of ignoring a trustful agent becomes zero, provided that there is a majority of truthful agents. Simulations show that when the coordinating node detects and isolates all the attackers, the model recovers and converges to the truthful model.

Several works have investigated tensor decompositions in the context of estimation and prediction problems. Specifically, a recent study explored Tensor Generalized Linear Models (TGLM), in which regression coefficients are assumed to have a tensor structure known as the Low Separation Rank (LSR) decomposition. This structure offers a more favourable bias-variance trade-off than other tensor structures of the same sized core tensor. While the study explored various aspects of the Low Separation Rank Tensor Generalized Linear Model (LSR-TGLM), some aspects of said model and the LSR structure are not well understood. In our work, we discuss some theoretical properties of the LSR-TGLM problem and the set of LSR-structured tensors. We establish an asymptotic consistency result for the LSR-TGLM problem, showing that its MLE estimates converge in probability to the originating LSR-structured coefficient tensor as the sample size increases. This involves proving new properties of LSR-structured tensors, particularly compactness of the set of LSR-structured tensors. We also present conditions for local identifiability of the LSR-structured tensor in an LSR-TGLM problem. Our findings clarify the relationship between the intrinsic degrees of freedom of the LSR-TGLM problem and the rank of the Fisher Information Matrix. Lastly, we provide asymptotic normality results of the MLE estimates, which can be useful for likelihood ratio tests in replication studies.

This paper considers the problem of estimating the principal eigenvector of a covariance matrix from independent and identically distributed data samples in streaming settings. The streaming rate of data in many contemporary applications can be high enough that a single processor cannot finish an iteration of existing methods for eigenvector estimation before a new sample arrives. This paper formulates and analyzes a distributed variant of the classical Krasulina's method (D-Krasulina) that can keep up with the high streaming rate of data by distributing the computational load across multiple processing nodes. The analysis shows that---under appropriate conditions---D-Krasulina converges to the principal eigenvector in an order-wise optimal manner; i.e., after receiving $M$ samples across all nodes, its estimation error can be $O(1/M)$. In order to reduce the network communication overhead, the paper also develops and analyzes a mini-batch extension of D-Krasulina, which is termed DM-Krasulina. The analysis of DM-Krasulina shows that it can also achieve order-optimal estimation error rates under appropriate conditions, even when some samples have to be discarded within the network due to communication latency. Finally, experiments are performed over synthetic and real-world data to validate the convergence behaviors of D-Krasulina and DM-Krasulina in high-rate streaming settings.

This paper leverages a space-time-modulated metamaterial (ST-MTM) antenna as a beamspace multiple-input multiple-output (MIMO) receiver, which facilitates the retrieval of information from multiple users transmitting distinct data on the same carrier frequency. In this scenario, the information received from each user, located in a specific direction, is mapped to a distinct harmonic frequency component within the received spectrum. The phase constant (beta) of each tunable composite right/left-handed (CRLH) unit cell in a leaky wave antenna (LWA) is periodically modulated in the time domain, generating harmonic frequencies around the carrier frequency of the received signal. By applying a proper time-modulated sequence to each unit cell, the power levels of the harmonics, extracted from one port of the receiver, reach their maximum values at different directions, enabling spatial-spectral mapping. The measured bit-error-rate (BER) results, obtained using a fabricated prototype as the receiver, confirm the successful retrieval of data concurrently sent by multiple users.

A computational experiment is deemed reproducible if the same data and methods are available to replicate quantitative results by any independent researcher, anywhere and at any time, granted that they have the required computing power. Such computational reproducibility is a growing challenge that has been extensively studied among computational researchers as well as within the signal processing and machine learning research community [1] , [2] .

This paper considers the problem of understanding the exit time for trajectories of gradient-related first-order methods from saddle neighborhoods under some initial boundary conditions. Given the ‘flat’ geometry around saddle points, first-order methods can struggle to escape these regions in a fast manner due to the small magnitudes of gradients encountered. In particular, while it is known that gradient-related first-order methods escape strict-saddle neighborhoods, existing analytic techniques do not explicitly leverage the local geometry around saddle points in order to control behavior of gradient trajectories. It is in this context that this paper puts forth a rigorous geometric analysis of the gradient-descent method around strict-saddle neighborhoods using matrix perturbation theory. In doing so, it provides a key result that can be used to generate an approximate gradient trajectory for any given initial conditions. In addition, the analysis leads to a linear exit-time solution for gradient-descent method under certain necessary initial conditions, which explicitly bring out the dependence on problem dimension, conditioning of the saddle neighborhood, and more, for a class of strict-saddle functions.

The advancement of future large‐scale wireless networks necessitates the development of cost‐effective and scalable security solutions. Specifically, physical layer (PHY) security has been put forth as a cost‐effective alternative to cryptographic mechanisms that can circumvent the need for explicit key exchange between communication devices. Herein, a space–time‐modulated digitally‐coded metamaterial (MTM) leaky wave antenna (LWA) is proposed that can enable PHY security by achieving the functionalities of directional modulation (DM) using a machine learning‐aided branch‐and‐bound (B&B) optimized coding sequence. Theoretically, it is first shown that the proposed space–time MTM antenna can achieve DM through both the spatial and spectral manipulation of the orthogonal frequency division multiplexing signal. Simulation results are then provided as proof‐of‐principle, demonstrating the applicability of the approach for achieving DM in various communication settings. Furthermore, a prototype of the proposed architecture controlled by a field‐programmable gate array is realized, which achieves DM via an optimized coding sequence carried out by the learning‐aided B&B algorithm corresponding to the states of the MTM LWA's unit cells. Experimental results confirm the theory behind the space–time‐modulated MTM LWA in achieving DM, which is observed via both the spectral harmonic patterns and bit error rate measurements.

The accuracy of many downstream machine learning algorithms is tied to the training data having uncorrelated features. With the modern-day data often being streaming in nature, geographically distributed, and having large dimensions, it is paramount to apply both uncorrelated feature learning and dimensionality reduction techniques in this scenario. Principal Component Analysis (PCA) is a state-of-the-art tool that simultaneously yields uncorrelated features and reduces data dimensions by projecting data onto the eigenvectors of the population covariance matrix. This paper introduces a novel algorithm called Consensus-DIstributEd Generalized Oja (C-DIEGO), which is based on Oja's method, to estimate the dominant eigenvector of a population covariance matrix in a distributed, streaming setting. The algorithm considers a distributed network of arbitrarily connected nodes without a central coordinator and assumes data samples continuously arrive at the individual nodes in a streaming manner. It is established in the paper that C-DIEGO can achieve an order-optimal convergence rate if nodes in the network are allowed to have enough consensus rounds per algorithmic iteration. Numerical results are also reported in the paper that showcase the efficacy of the proposed algorithm.

Fiber optic bundles are used in narrow-diameter medical and industrial instruments for acquiring images from confined locations. Images transmitted through these bundles contain only one pixel of information per fiber core and fail to capture information from the cladding region between cores. Both factors limit the spatial resolution attainable with fiber bundles. We show here that computational imaging (CI) can be combined with spectral coding to overcome these two fundamental limitations and improve spatial resolution in fiber bundle imaging. By acquiring multiple images of a scene with a high-resolution mask pattern imposed, up to 17 pixels of information can be recovered from each fiber core. A dispersive element at the distal end of the bundle imparts a wavelength-dependent lateral shift on light from the object. This enables light that would otherwise be lost at the inter-fiber cladding to be transmitted through adjacent fiber cores. We experimentally demonstrate this approach using synthetic and real objects. Using CI with spectral coding, object features 5× smaller than individual fiber cores were resolved, whereas conventional imaging could only resolve features at least 1.5× larger than each core. In summary, CI combined with spectral coding provides an approach for overcoming the two fundamental limitations of fiber optic bundle imaging.

A time-modulated retrodirective array (RDA) is proposed to achieve directional modulation (DM) for physical layer (PHY) security in wireless communication. The designed and fabricated RDA is based on the phase conjugation technique, in which the retransmitted signal is time-modulated by using pin diodes as RF switches. By feeding proper periodic sequences to the pin diodes-based switches, the resulting harmonic signals can be suppressed in the desired secure angle, while their levels rise and distort the retransmitted signal in other unwanted angles, thereby enabling PHY security. Measured bit-error-rate (BER) results by injecting an orthogonal frequency-division multiplexing (OFDM) signal to the LO port of the PHY secured RDA verify better security performance in comparison with a conventional RDA without time modulation.

Multi-camera networks are increasingly becoming pervasive in many monitoring and surveillance applications, and have attracted much attention in distributed systems with collaborative, real-time decision-making capabilities. While in-network data compression brings significant energy savings in camera nodes, signal representation using sparse approximations and overcomplete dictionaries have been shown to outperform traditional compression methods. In this work, an end-to-end and real-time solution is designed and implemented to enable energy-efficient and robust dictionary learning in distributed camera networks by leveraging the spatial correlation of the collected multimedia data. Traditional distributed dictionary learning relies on consensus-building algorithms, which involve communicating with neighboring nodes until convergence is achieved. Existing methods, however, do not exploit spatial correlations in camera networks for improved energy efficiency. In contrast, low-computational-complexity metrics are employed in this work to quantify and exploit the spatial correlation across camera nodes in a wireless network for efficient distributed dictionary learning and in-network image compression. The performance of the proposed approach is validated through extensive simulations on public datasets as well as via real-world experiments on a testbed composed of Raspberry Pi nodes.

Phantom Enhanced 911 (E911) calls are automatically generated 2 second calls, are a serious concern on cellular networks, and consume critical resources. As networks become increasingly complex, detecting and troubleshooting the causes of phantom E911 calls is becoming increasingly difficult. In this paper machine learning (ML) tools are used to analyze anonymized call detail record data collected by a major US telecom network service provider. The data is carefully pre-processed and encoded using an efficient encoding method. Classification algorithms K Nearest Neighbors (KNN) and Decision Trees (DTs) are then implemented to study correlations between device and network level features and a mobile device's ability to initiate phantom calls. Based on the results, this work also suggests certain policy changes for network operators that may decrease the high volume of phantom E911 calls or alleviate the pressure of phantom E911 calls on a cellular network.

Robust position reconstruction is paramount for enabling discoveries in astroparticle physics as backgrounds are significantly reduced by only considering interactions within the fiducial volume. In this work, we present for the first time a method for position reconstruction using a Bayesian network which provides per interaction uncertainties. We demonstrate the utility of this method with simulated data based on the XENONnT detector design, a dual-phase xenon time-projection chamber, as a proof-of-concept. The network structure includes variables representing the 2D position of the interaction within the detector, the number of electrons entering the gaseous phase, and the hits measured by each sensor in the top array of the detector. The precision of the position reconstruction (difference between the true and expectation value of position) is comparable to the state-of-the-art methods – an RMS of 0.69 cm, 0.09 of the sensor spacing, for the inner part of the detector (<60 cm) and 0.98 cm, 0.12 of the sensor spacing, near the wall of the detector (>60 cm). More importantly, the uncertainty of each interaction position was directly computed, which is not possible with other reconstruction methods. The method found a median 3-σ confidence region of 11 cm^2 for the inner part of the detector and 21 cm^2 near the wall of the detector. We found the Bayesian network framework to be well suited to the problem of position reconstruction. The performance of this proof-of-concept, even with several simplifying assumptions, shows that this is a promising method for providing per interaction uncertainty, which can be extended to energy reconstruction and signal classification.

The effectiveness of machine learning algorithms largely depends on the goodness of the representation of data. While the massiveness in dimension and amount of modern day data requires dimension reduction and feature extraction for efficient use of available computational resources, the use of un-correlated features is known to enhance the performance of such machine learning algorithms. Thus, an efficient representation learning approach should focus on dimension reduction as well as uncorrelated feature extraction. Even though Principal Component Analysis (PCA) and linear autoencoders are fundamental data processing tools largely used for dimension reduction, they can also be used to extract uncorrelated features when engineered properly. At the same time, factors like ever-increasing volume of data or inherently distributed data generation impede the use of existing centralized solutions for representation learning that require availability of data at a single location. This paper proposes two variants of an algorithm called FAST-PCA (Fast and exAct diSTributed PCA) based on a feedforward neural network-based system that learn data representations in a distributed setting such that they are reduced in dimension as well as have uncorrelated features. The proposed variants are meant to curb the communication overheads prevalent in the existing solutions and are shown to converge to the exact solutions at a linear rate. These claims are further supported by extensive numerical experiments.

The fundamental task of classification given a limited number of training data samples is considered for physicalsystems with known parametric statistical models. The standalone learning-based and statistical model-based classifiers face major challenges towards the fulfillment of the classification task using a small training set. Specifically, classifiers that solely rely on the physics-based statistical models usually suffer from their inability to properly tune the underlying unobservable parameters, which leads to a mismatched representation of the system’s behaviors. Learning-based classifiers, on the other hand, typically rely on a large number of training data from the underlying physical process, which might not be feasible in most practical scenarios. In this paper, a hybrid classification method—termed HyPhyLearn—is proposed that exploits both the physics-based statistical models and the learning-based classifiers. The proposed solution is based on the conjecture that HyPhyLearn would alleviate the challenges associated with the individual approaches of learning-based and statistical model-based classifiers by fusing their respective strengths. The proposed hybrid approach first estimates the unobservable model parameters using the available (suboptimal) statistical estimation procedures, and subsequently use the physics-based statistical models to generate synthetic data. Then, the training data samples are incorporated with the synthetic data in a learning-based classifier that is based on domain-adversarial training of neural networks. Specifically, in order to address the mismatch problem, the classifier learns a mapping from the training data and the synthetic data to a common feature space. Simultaneously, the classifier is trained to find discriminative features within this space in order to fulfill the classification task. Two case studies from communications systems (physical layer security and multi-user detection) are presented in order to highlight the usefulness of HyPhyLearn. Numerical results demonstrate that the proposed approach leads to major classification improvements in comparison to the existing standalone or hybrid classification methods.