Modeling of longitudinal cohort data typically involves complex temporal dependencies between multiple variables. There, the transformer architecture, which has been highly successful in language and vision applications, allows us to account for the fact that the most recently observed time points in an individual's history may not always be the most important for the immediate future. This is achieved by assigning attention weights to observations of an individual based on a transformation of their values. One reason why these ideas have not yet been fully leveraged for longitudinal cohort data is that typically, large datasets are required. Therefore, we present a simplified transformer architecture that retains the core attention mechanism while reducing the number of parameters to be estimated, to be more suitable for small datasets with few time points. Guided by a statistical perspective on transformers, we use an autoregressive model as a starting point and incorporate attention as a kernel-based operation with temporal decay, where aggregation of multiple transformer heads, i.e. different candidate weighting schemes, is expressed as accumulating evidence on different types of underlying characteristics of individuals. This also enables a permutation-based statistical testing procedure for identifying contextual patterns. In a simulation study, the approach is shown to recover contextual dependencies even with a small number of individuals and time points. In an application to data from a resilience study, we identify temporal patterns in the dynamics of stress and mental health. This indicates that properly adapted transformers can not only achieve competitive predictive performance, but also uncover complex context dependencies in small data settings.

Many rare diseases offer limited established treatment options, leading patients to switch therapies when new medications emerge. To analyze the impact of such treatment switches within the low sample size limitations of rare disease trials, it is important to use all available data sources. This, however, is complicated when the use of measurement instruments changes during the observation period, for example when instruments are adapted to specific age ranges. The resulting disjoint longitudinal data trajectories complicate the application of traditional modeling approaches like mixed-effects regression. We tackle this by mapping observations of each instrument to an aligned low-dimensional temporal trajectory, enabling longitudinal modeling across instruments. Specifically, we employ a set of variational autoencoder architectures to embed item values into a shared latent space for each time point. Temporal disease dynamics and treatment switch effects are then captured through a mixed-effects regression model applied to latent representations. To enable statistical inference, we present a novel statistical testing approach that accounts for the joint parameter estimation of mixed-effects regression and variational autoencoders. The methodology is applied to quantify the impact of treatment switches for patients with spinal muscular atrophy. Here, our approach aligns motor performance items from different measurement instruments for mixed-effects regression and maps estimated effects back to the observed item level to quantify the treatment switch effect. Our approach allows for model selection as well as for assessing effects of treatment switching. The results highlight the potential of modeling in joint latent representations for addressing small data challenges.

Accurate identification of CRISPR-Cas9 off-target sites is essential for the safety assessment of genome-editing-based therapies. While numerous in silico prediction tools have been developed, their comparative performance and practical utility in preclinical workflows remain incompletely defined. We performed a systematic benchmarking of 14 in silico CRISPR-Cas9 off-target prediction tools, including both standard approaches and machine learning-based models. The analysis was based on a curated dataset derived from the CRISPRoffT database, comprising 3,827 deep-sequenced genomic sites across 26 guide RNA/Cas9 combinations in human cells. Sites with indel frequencies ≥0.1% were operationally defined as true off-targets. We evaluated tool performance using score distributions, correlation with indel frequencies, precision-recall characteristics, recall among top-ranked candidate sites, and the effect of combining tools. All tools assigned higher scores to true off-target sites compared with nontarget sites, although substantial overlap between classes was observed. Correlation between prediction scores and indel frequencies was weak to moderate, indicating limited ability to predict editing magnitude. Precision-recall performance was moderate across all tools, reflecting inherent trade-offs between sensitivity and specificity. Recall increased with the number of predicted sites considered, reaching approximately 77% among the top 500 and up to 83% among the top 1,250 sites, but leaving a substantial fraction of true off-targets undetected. Combining tools yielded only modest improvements. Current in silico tools enable prioritization of CRISPR-Cas9 off-target candidates but remain limited in their ability to comprehensively identify and quantitatively predict off-target activity. Our findings highlight the importance of considering both ranking performance and candidate site coverage and support the use of combined computational and experimental strategies for robust off-target assessment in preclinical gene editing workflows.

While artificial neural networks excel in unsupervised learning of non-sparse structure, classical statistical regression techniques offer better interpretability, in particular when sparseness is enforced by ℓ_1 regularization, enabling identification of which factors drive observed dynamics. We investigate how these two types of approaches can be optimally combined, exemplarily considering two-photon calcium imaging data where sparse autoregressive dynamics are to be extracted. We propose embedding a vector autoregressive (VAR) model as an interpretable regression technique into a convolutional autoencoder, which provides dimension reduction for tractable temporal modeling. A skip connection separately addresses non-sparse static spatial information, selectively channeling sparse structure into the ℓ_1-regularized VAR. ℓ_1-estimation of regression parameters is enabled by differentiating through the piecewise linear solution path. This is contrasted with approaches where the autoencoder does not adapt to the VAR model. Having an embedded statistical model also enables a testing approach for comparing temporal sequences from the same observational unit. Additionally, contribution maps visualize which spatial regions drive the learned dynamics.

SUMMARY:Multimodal approaches are increasingly leveraged for integrating omics data with textual biological knowledge. Yet there is still no accessible, standardized framework that enables systematic comparison of omics representations with different text encoders within a unified workflow. We present mmContext, a lightweight and extensible multimodal embedding framework built on top of the open-source Sentence Transformers library. The software allows researchers to train or apply models that jointly embed omics and text data using any numeric representation stored in an AnnData.obsm layer and any text encoder available in Hugging Face. mmContext supports integration of diverse biological text sources and provides pipelines for training, evaluation, and data preparation. We train and evaluate models for a RNA-Seq and text integration task, and demonstrate their utility through zero-shot classification of cell types and diseases across four independent datasets. By releasing all models, datasets, and tutorials openly, mmContext enables reproducible and accessible multimodal learning for omics-text integration. AVAILABILITY AND IMPLEMENTATION:Pretrained checkpoints and full source code for our custom MMContextEncoder are available on Hugging Face huggingface.co/jo-mengr. The Python package github.com/mengerj/mmcontext provides the model implementation and training and evaluation scripts for custom training. The releases for the publication can be accessed via zenodo: adata_hf_datasets: doi.org/10.5281/zenodo.19185217 and mmContext: doi.org/10.5281/zenodo.19185493.

Animals can provide meaningful context for human single-cell data. To transfer information between species, we propose a deep learning approach that pre-trains a conditional variational autoencoder on animal data and transfers its final encoder layers to a human network architecture. Our approach then aligns latent spaces by leveraging data-level and model-learned similarities. We utilize this for label transfer and differential gene expression analysis in cross-species pairs of liver, adipose tissue, and glioblastoma datasets. Our results are robust even when gene sets differ, or datasets are small. Thus, we reliably exploit similarities between species to provide context for human single-cell data.

In a longitudinal clinical registry, different measurement instruments might have been used for assessing individuals at different time points. To combine them, we investigate deep learning techniques for obtaining a joint latent representation, to which the items of different measurement instruments are mapped. This corresponds to domain adaptation, an established concept in computer science for image data. Using the proposed approach as an example, we evaluate the potential of domain adaptation in a longitudinal cohort setting with a rather small number of time points, motivated by an application with different motor function measurement instruments in a registry of spinal muscular atrophy (SMA) patients. There, we model trajectories in the latent representation by ordinary differential equations (ODEs), where person-specific ODE parameters are inferred from baseline characteristics. The goodness of fit and complexity of the ODE solutions then allows to judge the measurement instrument mappings. We subsequently explore how alignment can be improved by incorporating corresponding penalty terms into model fitting. To systematically investigate the effect of differences between measurement instruments, we consider several scenarios based on modified SMA data, including scenarios where a mapping should be feasible in principle and scenarios where no perfect mapping is available. While misalignment increases in more complex scenarios, some structure is still recovered, even if the availability of measurement instruments depends on patient state. A reasonable mapping is feasible also in the more complex real SMA dataset. These results indicate that domain adaptation might be more generally useful in statistical modeling for longitudinal registry data.

Single-cell technologies generate increasingly complex and multi-layered datasets, increasing the need for analysis workflows that incorporate additional biological information. Pretrained language models, with access to large corpora of biomedical literature, promise to provide such additional context to complement data-based analyses, yet recent approaches largely focus on data-intrinsic tasks. Here we propose a framework for context-aware enrichment of single-cell RNA sequencing data by aligning data-derived and literature-derived representations in a shared embedding space. We represent cells as sentences derived from ranked gene expression and metadata, and construct structurally parallel datasets from PubMed titles and abstracts. Lightweight encoder-only language models are trained jointly on both sources to learn a common embedding space, thus integrating additional layers of information from biomedical literature. Analyzing the joint embedding space, we show that biomedical literature can be meaningfully aligned with single-cell profiles to enrich standard analysis workflows. The trained models achieve robust annotation, capture functional states such as cytotoxicity, and reveal disease associations from literature-aligned embeddings. In developmental data, incorporating temporal metadata enables capturing temporal transitions consistent with cell lineage trajectories, demonstrating the potential of knowledge-augmented embeddings as a generalizable and interpretable strategy for extending single-cell analysis pipelines. ### Competing Interest Statement The authors have declared no competing interest. Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), Project-ID 499552394 - SFB 1597

The emergence of breakthrough artificial intelligence (AI) techniques has led to a renewed focus on how small data settings, i.e., settings with limited information, can benefit from such developments. This includes societal issues such as how best to include under-represented groups in data-driven policy and decision making, or the health benefits of assistive technologies such as wearables. We provide a conceptual overview, in particular contrasting small data with big data, and identify common themes from exemplary case studies and application areas. Potential solutions are described in a more detailed technical overview of current data analysis and modelling techniques, highlighting contributions from different disciplines, such as knowledge-driven modelling from statistics and data-driven modelling from computer science. By linking application settings, conceptual contributions and specific techniques, we highlight what is already feasible and suggest what an agenda for fully leveraging small data might look like.

Ordinary differential equations (ODEs) can provide mechanistic models of temporally local changes of processes, where parameters are often informed by external knowledge. While ODEs are popular in systems modeling, they are less established for statistical modeling of longitudinal cohort data, e.g., in a clinical setting. Yet, modeling of local changes could also be attractive for assessing the trajectory of an individual in a cohort in the immediate future given its current status, where ODE parameters could be informed by further characteristics of the individual. However, several hurdles so far limit such use of ODEs, as compared to regression-based function fitting approaches. The potentially higher level of noise in cohort data might be detrimental to ODEs, as the shape of the ODE solution heavily depends on the initial value. In addition, larger numbers of variables multiply such problems and might be difficult to handle for ODEs. To address this, we propose to use each observation in the course of time as the initial value to obtain multiple local ODE solutions and build a combined estimator of the underlying dynamics. Neural networks are used for obtaining a low-dimensional latent space for dynamic modeling from a potentially large number of variables, and for obtaining patient-specific ODE parameters from baseline variables. Simultaneous identification of dynamic models and of a latent space is enabled by recently developed differentiable programming techniques. We illustrate the proposed approach in an application with spinal muscular atrophy patients and a corresponding simulation study. In particular, modeling of local changes in health status at any point in time is contrasted to the interpretation of functions obtained from a global regression. This more generally highlights how different application settings might demand different modeling strategies.

There are various dimensionality reduction techniques for visually inspecting dynamical patterns in time-series single-cell RNA-sequencing (scRNA-seq) data. However, the lack of one-to-one correspondence between cells across time points makes it difficult to uniquely uncover temporal structure in a low-dimensional manifold. The use of different techniques may thus lead to discrepancies in the representation of dynamical patterns. However, The extent of these discrepancies remains unclear. To investigate this, we propose an approach for reasoning about such discrepancies based on synthetic time-series scRNA-seq data generated by variational autoencoders. The synthetic dynamical patterns induced in a low-dimensional manifold reflect biologically plausible temporal patterns, such as dividing cell clusters during a differentiation process. We consider manifolds from different dimensionality reduction techniques, such as principal component analysis, t-distributed stochastic neighbor embedding, uniform manifold approximation, and projection and single-cell variational inference. We illustrate how the proposed approach allows for reasoning about to what extent low-dimensional manifolds, obtained from different techniques, can capture different dynamical patterns. None of these techniques was found to be consistently superior and the results indicate that they may not reliably represent dynamics when used in isolation, underscoring the need to compare multiple perspectives. Thus, the proposed synthetic dynamical pattern approach provides a foundation for guiding future methods development to detect complex patterns in time-series scRNA-seq data.

Dimensionality reduction greatly facilitates the exploration of cellular heterogeneity in single-cell RNA sequencing data. While most of such approaches are data-driven, it can be useful to incorporate biologically plausible assumptions about the underlying structure or the experimental design. We propose the boosting autoencoder (BAE) approach, which combines the advantages of unsupervised deep learning for dimensionality reduction and boosting for formalizing assumptions. Specifically, our approach selects small sets of genes that explain latent dimensions. As illustrative applications, we explore the diversity of neural cell identities and temporal patterns of embryonic development.

In settings requiring synthetic data generation based on a clinical cohort, e.g., due to data protection regulations, heterogeneity across individuals might be a nuisance that we need to control or faithfully preserve. The sources of such heterogeneity might be known, e.g., as indicated by sub-groups labels, or might be unknown and thus reflected only in properties of distributions, such as bimodality or skewness. We investigate how such heterogeneity can be preserved and controlled when obtaining synthetic data from variational autoencoders (VAEs), i.e., a generative deep learning technique that utilizes a low-dimensional latent representation. To faithfully reproduce unknown heterogeneity reflected in marginal distributions, we propose to combine VAEs with pre-transformations. For dealing with known heterogeneity due to sub-groups, we complement VAEs with models for group membership, specifically from propensity score regression. The evaluation is performed with a realistic simulation design that features sub-groups and challenging marginal distributions. The proposed approach faithfully recovers the latter, compared to synthetic data approaches that focus purely on marginal distributions. Propensity scores add complementary information, e.g., when visualized in the latent space, and enable sampling of synthetic data with or without sub-group specific characteristics. We also illustrate the proposed approach with real data from an international stroke trial that exhibits considerable distribution differences between study sites, in addition to bimodality. These results indicate that describing heterogeneity by statistical approaches, such as propensity score regression, might be more generally useful for complementing generative deep learning for obtaining synthetic data that faithfully reflects structure from clinical cohorts.

Animal data can provide meaningful context for human gene expression at the single-cell level. This context can improve cell-type or cell-state detection and clarify how well the animal models human biological processes. To achieve this, we propose a deep learning approach that identifies a unified latent space to map complex patterns between datasets. Specifically, we combine variational autoencoders with a data-level nearest neighbor search to align neural network architectures across species. We visualize commonalities by mapping cell samples into the latent space. The aligned latent representation facilitates information transfer in applications of liver, white adipose tissue, and glioblastoma cells from various animal models. We also identify genes that exhibit systematic differences and commonalities between species. The results are robust for small datasets and with large differences in the observed gene sets. Thus, we reliably uncover and exploit similarities between species to provide context for human single-cell data. ### Competing Interest Statement The authors have declared no competing interest.

SUMMARY:Several approaches have been proposed to reconstruct interactions between groups of cells or individual cells from single-cell transcriptomics data, leveraging prior information about known ligand-receptor interactions. To enhance downstream analyses, we present an end-to-end dimensionality reduction workflow, specifically tailored for single-cell cell-cell interaction data. In particular, we demonstrate that sparse dimensionality reduction can pinpoint specific ligand-receptor interactions in relation to clusters of cell pairs. For sparse dimensionality reduction, we focus on the Boosting Autoencoder approach. Overall, we provide a comprehensive workflow, including result visualization, that simplifies the analysis of interaction patterns in cell pairs. This is supported by a Jupyter notebook that can readily be adapted to different datasets. AVAILABILITY AND IMPLEMENTATION:https://github.com/NiklasBrunn/Sparse-dimension-reduction.

BACKGROUND:Time-series forecasting models play a central role in guiding intensive care coronavirus disease 2019 (COVID-19) bed capacity in a pandemic. A key predictor of future intensive care unit (ICU) COVID-19 bed occupancy is the number of new severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections in the general population, which in turn is highly associated with week-to-week variability, reporting delays, regional differences, number of unknown cases, time-dependent infection rates, vaccinations, SARS-CoV‑2 virus variants, and nonpharmaceutical containment measures. Furthermore, current and also future COVID ICU occupancy is significantly influenced by ICU discharge and mortality rates. METHODS:Both the number of new SARS-CoV‑2 infections in the general population and intensive care COVID-19 bed occupancy rates are recorded in Germany. These data are statistically analyzed on a daily basis using epidemic SEIR (susceptible, exposed, infection, recovered) models using ordinary differential equations and multiple regression models. RESULTS:Forecast results of the immediate trend (20-day forecast) of ICU occupancy by COVID-19 patients are made available to decision makers at various levels throughout the country. CONCLUSION:The forecasts are compared with the development of available ICU bed capacities in order to identify capacity limitations at an early stage and to enable short-term solutions to be made, such as supraregional transfers.

Like many groups considering the new programming language Julia, we faced the challenge of accessing the algorithms that we develop in Julia from R. Therefore, we developed the R package JuliaConnectoR, available from the Comprehensive R Archive Network (CRAN), the official R package repository, and from GitHub (https://github. com/stefan-m-lenz/JuliaConnectoR), in particular for making advanced deep learning tools available. For maintainability and stability, we decided to base communication between R and Julia on the transmission control protocol, using an optimized binary format for exchanging data. Our package also specifically contains features that allow for a convenient interactive use in R. This makes it easy to develop R extensions with Julia or to simply call functionality from Julia packages in R. Interacting with Julia objects and calling Julia functions becomes user-friendly, as Julia functions and variables are made directly available as objects in the R workspace. We illustrate the further features of our package with code examples, and also discuss advantages over the two alternative packages JuliaCall and XRJulia. Finally, we demonstrate the usage of the package with a more extensive example for employing neural ordinary differential equations, a recent deep learning technique that has received much attention. This example also provides more general guidance for integrating deep learning techniques from Julia into R.

Recent extensions of single-cell studies to multiple data modalities raise new questions regarding experimental design. For example, the challenge of sparsity in single-omics data might be partly resolved by compensating for missing information across modalities. In particular, deep learning approaches, such as deep generative models (DGMs), can potentially uncover complex patterns via a joint embedding. Yet, this also raises the question of sample size requirements for identifying such patterns from single-cell multi-omics data. Here, we empirically examine the quality of DGM-based integrations for varying sample sizes. We first review the existing literature and give a short overview of deep learning methods for multi-omics integration. Next, we consider eight popular tools in more detail and examine their robustness to different cell numbers, covering two of the most common multi-omics types currently favored. Specifically, we use data featuring simultaneous gene expression measurements at the RNA level and protein abundance measurements for cell surface proteins (CITE-seq), as well as data where chromatin accessibility and RNA expression are measured in thousands of cells (10x Multiome). We examine the ability of the methods to learn joint embeddings based on biological and technical metrics. Finally, we provide recommendations for the design of multi-omics experiments and discuss potential future developments.