Recent deep neural network architectures that are tailored to tabular data operate at the feature level and process multiple latent representations simultaneously, typically one per feature. We investigate the impact of varying the dimension and number of such latent representations on model performance and generalization. Our results identify distinct model behaviors during both training and testing phases. To ease analysis of these behaviors, we propose a novel tool for characterizing data complexity and use it to highlight intricate relationships between data complexity, model complexity and model performance. We hypothesize a phenomenon of implicit self-regularization which intensifies with model capacity and sample-to-dimension ratio. While this self-regularization can mitigate over-fitting, it may also lead to reduced performance on training data. Our findings expand the understanding of neural networks applied to tabular data and provide insights that can help practitioners and/or automated methods in designing neural networks architectures that better match the complexity of specific tabular data sets.

Student t-distributed stochastic neighbor embedding (t-SNE) finds low-dimensional data representations allowing visual exploration of data sets.t-SNE minimises a cost function with a custom two-phase gradient descent.The first phase is called early exaggeration and involves a hyper-parameter whose value can be tricky and time-consuming to set.This paper proposes another way to optimise the cost function without early exaggeration.Empirical evaluation shows that the proposed method of optimization converges faster and yields competitive results in terms of neighborhood preservation.

Dimensionality reduction techniques are essential in current single-cell ‘omics approaches, offering biologists a first glimpse of the structure present in their data. These methods are most often used to visualise high-dimensional and noisy input datasets, but are also frequently applied for downstream structure learning. By design, every dimensionality reduction technique preserves some characteristics of the original, high-dimensional data, while discarding others. We introduce ViScore , a framework for validation of low-dimensional embeddings, consisting of novel quantitative measures and visualisations to assess their quality in both supervised and unsupervised settings. Next, we present ViVAE , a new dimensionality reduction method which uses graph-based transformations and deep learning models to visualise important structural relationships. We demon-strate that ViVAE strikes a better balance in preserving both local and global structures compared to existing methods, achieving general-purpose visualisation but also facilitating analyses of developmental trajectories.

Most recent deep neural network architectures for tabular data operate at the feature level and process multiple latent representations simultaneously.While the dimension of these representations is set through hyper-parameter tuning, their number is typically fixed and equal to the number of features in the original data.In this paper, we explore the impact of varying the number of latent representations on model performance.Our results suggest that increasing the number of representations beyond the number of features can help capture more complex interactions, whereas reducing their number can improve performance in cases where there are many uninformative features.

Multidimensional scaling is a statistical process that aims to embed high dimensional data into a lower-dimensional space; this process is often used for the purpose of data visualisation. Common multidimensional scaling algorithms tend to have high computational complexities, making them inapplicable on large data sets. This work introduces a stochastic, force directed approach to multidimensional scaling with a time and space complexity of O(N), with N data points. The method can be combined with force directed layouts of the family of neighbour embedding such as t-SNE, to produce embeddings that preserve both the global and the local structures of the data. Experiments assess the quality of the embeddings produced by the standalone version and its hybrid extension both quantitatively and qualitatively, showing competitive results outperforming state-of-the-art approaches. Codes are available at https://github.com/PierreLambert3/SQuaD-MDS-and-FItSNE-hybrid.

Nonlinear dimensionality reduction (NLDR) algorithms such as 𝑡 -SNE are often employed to visually analyze high-dimensional (HD) data sets in the form of low-dimensional (LD) embeddings. Unfortunately, the nonlinearity of the NLDR process prohibits the interpretation of the resulting embeddings in terms of the HD features. State-of-the-art studies propose post-hoc explanation approaches to locally explain the embeddings. However, such tools are typically slow and do not automatically cover the entire LD embedding, instead providing local explanations around one selected data point at a time. This prevents users from quickly gaining insights about the general explainability landscape of the embedding. This paper presents a globally local and fast explanation framework for NLDR embeddings. This framework is fast because it only requires the computation of sparse linear regression models on subsets of the data, without ever reapplying the NLDR algorithm itself. In addition, the framework is globally local in the sense that the entire LD embedding is automatically covered by multiple local explanations. The different interpretable structures in the embedding are directly characterized, making it possible to quantify the importance of the HD features in various regions of the LD embedding. An example use-case is examined, emphasizing the value of the presented framework. Public codes and a software are available at https://github.com/PierreLambert3/glocally_explained.

Multidimensional scaling is a statistical process that aims to embed high-dimensional data into a lower-dimensional, more manageable space.Common MDS algorithms tend to have some limitations when facing large data sets due to their high time and spatial complexities.This paper attempts to tackle the problem by using a stochastic approach to MDS which uses gradient descent to optimise a loss function defined on randomly designated quartets of points.This method mitigates the quadratic memory usage by computing distances on the fly, and has iterations in O(N ) time complexity, with N samples.Experiments show that the proposed method provides competitive results in reasonable time.Public codes are available at https://github.com/PierreLambert3/SQuaD-MDS.git. Multidimensional scaling and its limitationsDimensionality reduction (DR) is the process of mapping high-dimensional (HD) observations into a lower-dimensional (LD) space such that the LD embedding is a faithful representation of the HD data.The main DR uses are in machine learning, to curb the curse of dimensionality, and in visualisation.Mapped data can reveal structures that would lay hidden from the human perception if left in HD.Typically, some information is lost by the DR and, therefore, each DR method has a take on what kind of information should be preserved and what can be lost.Used frequently in visualisation, t-SNE [1] aims at retaining the neighbourhood of each point according to a distance metric and a perplexity, which reflects the size of the neighbourhood to preserve.While t-SNE excels at retaining local structures, sufficiently remote points tend to be considered equally distant by the algorithm and, therefore, the larger-scale structures can be distorted.Such distortions can lead to erroneous conclusions by the human user, who might overestimate the dissimilarity between two clusters that are distant in the LD embedding.For this reason, using multiple DR paradigms in conjunction is a good practice in visualisation: another embedding that preserves distances instead of neighbourhoods would have prevented this erroneous conclusion.This paper considers metric multidimensional scaling (MDS): a DR technique that produces a LD embedding such that the pairwise distances in LD reflect those in HD.MDS minimises a cost function which, in its simplest form, is the sum of the squared differences between distances in HD and the Euclidean distances in LD.A common strategy to optimize this cost function is based on 417