Parametric models deployed in non-stationary environments degrade as the underlying data distribution evolves over time (a phenomenon known as temporal domain drift). In the current work, we present KOMET (Koopman Operator identification of Model parameter Evolution under Temporal drift), a model-agnostic, data-driven framework that treats the sequence of trained parameter vectors as the trajectory of a nonlinear dynamical system and identifies its governing linear operator via Extended Dynamic Mode Decomposition (EDMD). A warm-start sequential training protocol enforces parameter-trajectory smoothness, and a Fourier-augmented observable dictionary exploits the periodic structure inherent in many real-world distribution drifts. Once identified, KOMET's Koopman operator predicts future parameter trajectories autonomously, without access to future labeled data, enabling zero-retraining adaptation at deployment. Evaluated on six datasets spanning rotating, oscillating, and expanding distribution geometries, KOMET achieves mean autonomous-rollout accuracies between 0.981 and 1.000 over 100 held-out time steps. Spectral and coupling analyses further reveal interpretable dynamical structure consistent with the geometry of the drifting decision boundary.

Recently, there has been a growing demand for advances in representation learning for graphs. The literature has developed methods to represent nodes in an embedding space, allowing for classical techniques to perform node classification and prediction. One such method is the graph convolutional neural network that aggregates the node neighbor's features to create the embedding. In this method, the embedding contains local information about an individual's connections but lacks the global community dynamics about that individual. We propose a method that leverages both local and global information, offering significant advancements in the analysis of social networks. We first represent information across the entire hierarchy of the network by allowing the graph convolutional network to skip neighbors in its convolutions. We propose 3 methods of skipping that leverage matrix-powers of the adjacency matrix and a breadth-first search traversal. Once convolutions are performed, we capture correlations across the hierarchies by constructing our convolutions into a tensor (e.g., multi-way array), enabling a more holistic understanding of individual nodes' roles within their communities. We present experimental results for the proposed method and compare/contrast with other state-of-the-art methods in benchmark social network datasets for node classification and link prediction tasks. Ultimately, the proposed method not only advances the field of graph representation learning but also demonstrates improved performance across various complex social networks.

In the era of big data, there are increasing demands to forecast incomplete, sparse, and nonstationary data. The current research aims to solve these problems for multilinear time series through a combination of tensor autoregression and low-rank tensor factorization. We propose an expansion of TMF to second-order data: temporal tensor factorization (TTF). The current research aims to interpolate missing values via low-rank tensor factorization, which produces a latent time series. We perform forecasting in the latent space with a transform-based tensor autoregression (L-TAR) process. We present experimental results of the proposed method with other state-of-the-art methods on the benchmark datasets which include video imaging, dynamic networks, and energy usage.

Representing videos as linear subspaces on Grassmann manifolds has made great strides in action recognition problems. Recent studies have explored the convenience of discriminant analysis by making use of Grassmann kernels. However, traditional methods rely on the matrix representation of videos based on the temporal dimension and suffer from not considering the two spatial dimensions. To overcome this problem, we keep the natural form of videos by representing video inputs as multidimensional arrays known as tensors and propose a tensor discriminant analysis approach on Grassmannian manifolds. Because matrix algebra does not handle tensor data, we introduce a new Grassmann projection kernel based on the tensor-tensor decomposition and product. Experiments with human action databases show that the proposed method performs well compared with the state-of-the-art algorithms.

In the era of big data, there is massive demand for new techniques to forecast and analyze multi-dimensional data. One task that has seen great interest in the community is anomaly detection of streaming data. Toward this end, the current research develops a novel approach to anomaly detection of streaming 2-dimensional observations via multilinear timeseries analysis and 3-dimensional tensor principal component analysis (3DTPCA). We approach this problem utilizing dimensionality reduction and probabilistic inference in a lowdimensional space. We first propose a natural extension to 2dimensional tensor principal component analysis (2DTPCA) to perform data dimensionality reduction on 4-dimensional tensor objects, aptly named 3DTPCA. We then represent the subsequences of our time-series observations as a 4-dimensional tensor utilizing a sliding window. Finally, we use 3DTPCA to compute reconstruction errors for inferring anomalous instances within the multilinear data stream. Experimental validation is presented via MovingMNIST data. Results illustrate that the proposed approach has a significant speedup in training time compared with deep learning, while performing competitively in terms of accuracy.

In the era of big data, there is an increasing demand for new methods for analyzing and forecasting 2-dimensional data. The current research aims to accomplish these goals through the combination of time-series modeling and multilinear algebraic systems. We expand previous autoregressive techniques to forecast multilinear data, aptly named the L-Transform Tensor autoregressive (L-TAR for short). Tensor decompositions and multilinear tensor products have allowed for this approach to be a feasible method of forecasting. We achieve statistical independence between the columns of the observations through invertible discrete linear transforms, enabling a divide and conquer approach. We present an experimental validation of the proposed methods on datasets containing image collections, video sequences, sea surface temperature measurements, stock prices, and networks.

Multilinear discriminantanalysis (MLDA), a novel approach based upon recent developments in tensor-tensor decomposition, has been proposed recently and showed better performance than traditional matrix linear discriminant analysis (LDA). The current paper presents a nonlinear generalization of MLDA (referred to as KMLDA) by extending the well known "kernel trick" to multilinear data. The approach proceeds by defining a new dot product based on new tensor operators for third-order tensors. Experimental results on the ORL, extended Yale B, and COIL-100 data sets demonstrate that performing MLDA in feature space provides more class separability. It is also shown that the proposed KMLDA approach performs better than the Tucker-based discriminant analysis methods in terms of image classification.

The current paper presents a new approach to multilinear dynamical systems analysis and control. The approach is based upon recent developments in tensor decompositions and a newly defined algebra of circulants. In particular, it is shown that under the right tensor multiplication operator, a third order tensor can be written as a product of third order tensors that is analogous to a traditional matrix eigenvalue decomposition where the "eigenvectors" become eigenmatrices and the "eigenvalues" become eigen-tuples. This new development allows for a proper tensor eigenvalue decomposition to be defined and has natural extension to linear systems theory through a \textit{tensor-exponential}. Through this framework we extend many of traditional techniques used in linear system theory to their multilinear counterpart.

Two-directional two-dimensional principal component analysis ((2D$)^{2}$PCA) has shown promising results for it’s ability to both represent and recognize facial images. The current paper extends these results into a multilinear framework (referred to as two-directional Tensor PCA or 2DTPCA for short) using a recently defined tensor operator for 3 rd -order tensors. The approach proceeds by first computing a low-dimensional projection tensor for the row-space of the image data (generally referred to as mode-l) and then subsequently computing a low-dimensional projection tensor for the column space of the image data (generally referred to as mode-3). Experimental results are presented on the ORL, extended Yale-B, COIL100, and MNIST data sets that show the proposed approach outperforms traditional “ tensor-based” PCA approaches with a much smaller subspace dimension in terms of recognition rates.

Forecasting, especially high-dimensional forecasting, is becoming more and more sought after, particularly as computing resources increase in both size and speed. Flow field forecasting is a general purpose regression-based forecasting method that has recently been expanded to high-dimensional settings. In this article, we provide an overview of the flow field forecasting methodology, with a particular emphasis on environments where the number of candidate predictor variables is large, potentially larger than the number of observations.

The tensor singular value decomposition (t-SVD) based on an algebra of circulants is an effective multilinear sub- space learning technique for dimensionality reduction and data classification. Unfortunately, the computational cost associated with computing the t-SVD can become prohibitively expensive, particularly when dealing with very large data sets. In this paper, we present a computationally efficient approach for estimating the t-SVD by capitalizing on the correlations of the data in the temporal dimension. The approach proceeds by extending our prior work on fast eigenspace decompositions by transforming the tensor data from the spatial domain to the spectral domain in order to obtain reduced order harmonic tensor. The t-SVD can then be applied in the transform domain thereby significantly reducing the computational burden. Experimental results which are presented on the extended Yale-B, COIL-100, and MNIST data sets show the proposed method provides considerable computational savings with the approximated subspaces that are nearly the same as the true subspaces as computed via the t-SVD.

With the massive influx of 2-dimensional observational data, new methods for analyzing, modeling, and forecasting multidimensional data need to be developed. The current research aims to accomplish these goals through the intersection of time-series modeling and multi-linear algebraic systems. In particular, the current research, aptly named the $\mathcal{L}$-Transform Tensor Auto-Regressive ($\mathcal{L}$-TAR for short) model expands previous auto-regressive techniques to forecast data from multilinear observations as oppose to scalars or vectors. The approach is based on recent developments in tensor decompositions and multilinear tensor products. Transforming the multilinear data through invertible discrete linear transforms enables statistical Independence between observations. As such, can be reformulated to a collection of vector auto-regression problems for model learning. Experimental results are provided on benchmark datasets containing image collections, video sequences, sea surface temperature measurements, and stock closing prices.

t-distributed Stochastic Neighbor Embedding (t-SNE) is a data visualization tool that was developed to provide a flexible, nonparametric method for mapping high dimensional data onto a two or three dimensional subspace for data visualization. This paper observes the effects of using different intermediate data reduction algorithms (e.g., Principal Component Analysis, Independent Component Analysis, Linear Discriminant Analysis, Sammon Mapping, and Local Linear Embedding) to first reduce the data to an intermediate subspace prior to applying t-SNE for visualization. Our research shows that no intermediate step in the visualization process is trivial, and application dependent knowledge should be utilized to ensure the best possible visualization in lower dimensional spaces. Experimental results are presented for several common data sets where we illustrate that, for clustering applications and visualization of class separation of multi-class data, each algorithm tested results in significantly different mappings.

Binary decision trees are non-linear prediction models dating back to Breiman's work in the 1980's. The standard algorithm for building a decision tree is a greedy approach whereby all variables and levels of the variable are cycled through until the partition is found that minimizes the impurity of the partition. Our approach delays the split decision one or more steps. We have found that delaying the split decision just one level often provides a tree with better statistical properties and better predictive capability.

Flow field (FF) forecasting is a statistical framework for generalpurpose time series forecasting that can be readily adapted to various applications. Given a historical time series space, FF forecasting can actively search the space and determine which variables are most useful in prediction. FF forecasting was first developed as a univariate forecasting technique, but was extended to bivariate time series. In this paper we show that FF forecasting can further be extended to higher dimensional time series involving potentially hundreds of predictor variables. We call this implementation Tree based-flow field (TB-FF) forecasting. Using a tree-based algorithm, we sift through the predictor space in order to find the best predictor variables in a large candidate pool. We show that TB-FF forecasting technique can outperform many of the traditional techniques, especially when the time series data is non-stationary.

Advanced decision making using machine learning should be both accurate and interpretable. Many standard machine learning techniques suffer from an inherent lack of transparency with regard to how the resulting decision was made. In the current work we aim to overcome this issue by introducing a hybrid learning approach using classical decision trees alongside artificial neural networks, dubbed a "neural shrub". The Neural Shrub methodology presented in this paper aims to maintain as much interpretability as possible without sacrificing either classification or regression accuracy. Experimental results are presented on several benchmark data sets to validate the proposed approach as well as provide insight into future research directions.