Fighting games such as Street Fighter II present unique challenges to reinforcement learning (RL) agents due to their fast-paced, real-time nature. In most RL frameworks, agents are hard-coded to make decisions at a fixed interval, typically every frame or every N frames. Although this design ensures timely responses, it restricts the agent's ability to adjust its reaction timing. Acting every frame grants frame-perfect reflexes, which are unrealistic compared to human players, whereas longer fixed intervals reduce computational cost but hinder responsiveness. We consider an alternative decision-making framework in which the agent learns not only what action to take but also for how long to execute it. By jointly predicting both action and duration, the agent can dynamically adapt its responsiveness to different situations in the game. We implement this method using the open-source FightLadder environment with agents trained against scripted built-in bots, systematically testing different frame skip configurations to analyze their influence on performance, responsiveness, and learned behavior. Experiments show that learned timing can match the performance of well-chosen fixed frame skips and encourages repeatable action patterns, but does not ensure robustness on its own. In most cases, we see agents performing best with consistently high frame skip values (i.e., low responsiveness). This strategy makes it easier to learn exploitative strategies where the same action is repeated over and over, which the scripted bots appear to be susceptible to.

It is common practice in reinforcement learning (RL) research to train and deploy agents in bespoke simulators, typically implemented by engineers directly in general-purpose programming languages or hardware acceleration frameworks such as CUDA or JAX. This means that programming and engineering expertise is not only required to develop RL algorithms, but is also required to use already developed algorithms for novel problems. The latter poses a problem in terms of the usability of RL, in particular for private individuals and small organisations without substantial engineering expertise. We also perceive this as a challenge for effective generalisation in RL, in the sense that is no standard, shared formalism in which different problems are represented. As we typically have no consistent representation through which to provide information about any novel problem to an agent, our agents also cannot instantly or rapidly generalise to novel problems. In this position paper, we advocate for a research agenda centred around the use of user-friendly description languages for describing problems, such that (i) users with little to no engineering expertise can formally describe the problems they would like to be tackled by RL algorithms, and (ii) algorithms can leverage problem descriptions to effectively generalise among all problems describable in the language of choice.

The transition to the High-Luminosity Large Hadron Collider (HL-LHC) presents a computational challenge where particle reconstruction complexity may outpace classical computing resources. While quantum computing offers potential speedups, standard algorithms like Harrow-Hassidim-Lloyd (HHL) require prohibitive circuit depths for near-term hardware. Here, we introduce a 1-Bit Quantum Filter, a domain-specific adaptation of HHL that reformulates tracking from matrix inversion to binary ground-state filtering. By replacing high-precision phase estimation with a single-ancilla spectral threshold and exploiting the Hamiltonian’s sparsity, we achieve an asymptotic gate complexity of $${{\mathscr{O}}}(\sqrt{N}\log N)$$, given Hamiltonian dimension N. We validate this approach on LHCb Monte Carlo events, demonstrating segment finding efficiency highly competitive with the classical state-of-the-art methods. Furthermore, we benchmark performance using the Quantinuum System Model H2 trapped-ion processor and IBM Heron R3 superconducting processor. This work establishes a quantum track reconstruction method capable of solving realistic event topologies on noise-free simulators and smaller tracking scenarios within the current constraints of the Noisy Intermediate Scale Quantum (NISQ) era. Remaining challenges toward a full end-to-end tracking solution include an efficient readout and Hamiltonian construction. Trajectory reconstruction of elementary particles in future high-energy collider experiments is a highly complex combinatorial problem. A 1-Bit quantum filter, presented here, is a candidate to be applied in a Quantum-Classical workflow. Here we show the algorithm’s performance on quantum hardware and noise-free simulators.

In the future high-luminosity LHC era, high-energy physics experiments face unprecedented computational challenges for event reconstruction. Employing the LHCb vertex locator as a case study we investigate a novel approach for charged particle track reconstruction. The algorithm hinges on minimizing an Ising-like Hamiltonian using matrix inversion. Solving this matrix inversion classically achieves reconstruction efficiencies akin to current stateof-the-art algorithms. Exploiting the Harrow-Hassidim-Lloyd (HHL) quantum algorithm for linear systems holds the promise of an exponential speedup in the number of input hits over its classical counterpart, contingent on the conditions of efficient quantum phase estimation (QPE) and effectively reading out the algorithm’s output. This contribution builds on previous work by Nicotra et al. [1] and strives to fulfill these conditions and further streamlines the algorithm’s circuit depth by a factor up to 104. Our version of the HHL algorithm restricts the QPE precision to one bit, largely reducing circuit depth and addressing HHL’s readout issue. Furthermore, this allows for the implementation of a post-processing algorithm that reconstructs event Primary Vertices (PVs). The findings presented here aim to further illuminate the potential of harnessing quantum computing for the future of particle track reconstruction in high-energy physics.

Electrocardiographic imaging (ECGI) aims to noninvasively estimate heart surface potentials starting from body surface potentials. This is classically based on geometric information on the torso and the heart from imaging, which complicates clinical application. In this study, we aim to develop a deep learning framework to estimate heart surface potentials solely from body surface potentials, enabling wider clinical use. The framework introduces two main components: the transformation of 3D torso and heart geometries into standard 2D representations, and the development of a customized deep learning network model. The 2D torso and heart representations maintain a consistent layout across different subjects, making the proposed framework applicable to different torso-heart geometries. With spatial information incorporated in the 2D representations, the torso-heart physiological relationship can be learnt by the network. The deep learning model is based on a Pix2Pix network, adapted to work with 2.5D data in our task, i.e., 2D body surface potential maps (BSPMs) and 2D heart surface potential maps (HSPMs) with time sequential information. We propose a new loss function tailored to this specific task, which uses a cosine similarity and different weights for different inputs. BSPMs and HSPMs from 11 healthy subjects (8 females and 3 males) and 29 idiopathic ventricular fibrillation (IVF) patients (11 females and 18 males) were used in this study. Performance was assessed on a test set by measuring the similarity and error between the output of the proposed model and the solution provided by mainstream ECGI, by comparing HSPMs, the concatenated electrograms (EGMs), and the estimated activation time (AT) and recovery time (RT). The mean of the mean absolute error (MAE) for the HSPMs was 0.012 ± 0.011, and the mean of the corresponding structural similarity index measure (SSIM) was 0.984 ± 0.026. The mean of the MAE for the EGMs was 0.004 ± 0.004, and the mean of the corresponding Pearson correlation coefficient (PCC) was 0.643 ± 0.352. Results suggest that the model is able to precisely capture the structural and temporal characteristics of the HSPMs. The mean of the absolute time differences between estimated and reference activation times was 6.048 ± 5.188 ms, and the mean of the absolute differences for recovery times was 18.768 ± 17.299 ms. Overall, results show similar performance between the proposed model and standard ECGI, exhibiting low error and consistent clinical patterns, without the need for CT/MRI. The model shows to be effective across diverse torso-heart geometries, and it successfully integrates temporal information in the input. This in turn suggests the possible use of this model in cost effective clinical scenarios like patient screening or post-operative follow-up.

Quantum Computing is a rapidly developing field with the potential to tackle the increasing computational challenges faced in high-energy physics. In this work, we explore the potential and limitations of variational quantum algorithms in solving the particle track reconstruction problem. We present an analysis of two distinct formulations for identifying straight-line tracks in a multilayer detection system, inspired by the LHCb vertex detector. The first approach is formulated as a ground-state energy problem, while the second approach is formulated as a system of linear equations. This work addresses one of the main challenges when dealing with variational quantum algorithms on general problems, namely designing an expressive and efficient quantum ansatz working on tracking events with fixed detector geometry. For this purpose, we employed a quantum architecture search method based on Monte Carlo Tree Search to design the quantum circuits for different problem sizes. We provide experimental results to test our approach on both formulations for different problem sizes in terms of performance and computational cost.

It is common practice in reinforcement learning (RL) research to train and deploy agents in bespoke simulators, typically implemented by engineers directly in general-purpose programming languages or hardware acceleration frameworks such as CUDA or JAX. This means that programming and engineering expertise is not only required to develop RL algorithms, but is also required to use already developed algorithms for novel problems. The latter poses a problem in terms of the usability of RL, in particular for private individuals and small organisations without substantial engineering expertise. We also perceive this as a challenge for effective generalisation in RL, in the sense that is no standard, shared formalism in which different problems are represented. As we typically have no consistent representation through which to provide information about any novel problem to an agent, our agents also cannot instantly or rapidly generalise to novel problems. In this position paper, we advocate for a research agenda centred around the use of user-friendly description languages for describing problems, such that (i) users with little to no engineering expertise can formally describe the problems they would like to be tackled by RL algorithms, and (ii) algorithms can leverage problem descriptions to effectively generalise among all problems describable in the language of choice.

Many enhancements to Monte-Carlo Tree Search (MCTS) have been proposed over almost two decades of general game playing and other artificial intelligence research. However, our ability to characterise and understand which variants work well or poorly in which games is still lacking. This paper describes work on an initial dataset that we have built to make progress towards such an understanding: 268,386 plays among 61 different agents across 1494 distinct games. We describe a preliminary analysis and work on training predictive models on this dataset, as well as lessons learned and future plans for a new and improved version of the dataset.

Gravitational-wave (GW) interferometers are able to detect a change in distance of $\sim$ 1/10,000th the size of a proton. Such sensitivity leads to large appearance rates of non-Gaussian transient noise bursts in the main detector strain, also known as glitches. These glitches come in a wide range of frequency-amplitude-time morphologies and are caused by environmental or instrumental processes, hindering searches for all sources of gravitational waves. Current approaches for their identification use supervised models to learn their morphology in the main strain, but do not consider relevant information provided by auxiliary channels that monitor the state of the interferometers nor provide a flexible framework for novel glitch morphologies. In this work, we present an unsupervised algorithm to find anomalous glitches. We encode a subset of auxiliary channels from LIGO Livingston in the fractal dimension, a measure for the complexity of the data, and learn the underlying distribution of the data using an auto-encoder with periodic convolutions. In this way, we uncover unknown glitch morphologies, and overlaps in time between different glitches and misclassifications. This led to the discovery of anomalies in $6.6 \%$ of the input data. The results of this investigation stress the learnable structure of auxiliary channels encoded in fractal dimension and provide a flexible framework to improve the state-of-the-art of glitch identification algorithms.

ABSTRACTIn this research, we present an alternative methodology to search for ring-like structures in the sky with unusually large temperature gradients, namely, Hawking points (HPs), in the Cosmic Microwave Background (CMB), which are possible observational effects associated with Conformal Cyclic Cosmology (CCC). To assess the performance of our method, we constructed an artificial data set of HP, according to CCC, and we were able to retrieve $95 {{\ \rm per\ cent}}$ of ring-like anomalies from it. Furthermore, we scanned the Planck CMB sky map and compared it to simulations according to ΛCDM, where we applied robust statistical tests to assess the existence of HP. Even though no significant ring-like structures were observed, we report the largest excess of HP candidates found at α =1 per cent significance level for the analysed sky maps (CMB at 70GHz, SEVEM, SMICA, and Commander-Ruler), and we stress the need to continue the theoretical and experimental research in this direction.

High-energy physics is facing increasingly demanding computational challenges in real-time event reconstruction for the near-future high-luminosity era. Using the LHCb vertex detector as a use case, we explore a new algorithm for particle track reconstruction based on the minimisation of an Ising-like Hamiltonian with a linear algebra approach. The use of a classical matrix inversion technique results in tracking performance similar to the current state-of-the-art but with worse scaling complexity in time. To solve this problem, we also present an implementation as a quantum algorithm, using the Harrow-Hassadim-Lloyd (HHL) algorithm: this approach can potentially provide an exponential speedup as a function of the number of input hits over its classical counterpart, in spite of limitations due to the well-known HHL Hamiltonian simulation and readout problems. The findings presented in this paper shed light on the potential of leveraging quantum computing for real-time particle track reconstruction in high-energy physics.

The liver is the primary site for the metabolism and detoxification of many compounds, including pharmaceuticals. Consequently, it is also the primary location for many adverse reactions. As the liver is not readily accessible for sampling in humans; rodent or cell line models are often used to evaluate potential toxic effects of a novel compound or candidate drug. However, relating the results of animal and in vitro studies to relevant clinical outcomes for the human in vivo situation still proves challenging. In this study, we incorporate principles of transfer learning within a deep artificial neural network allowing us to leverage the relative abundance of rat in vitro and in vivo exposure data from the Open TG-GATEs data set to train a model to predict the expected pattern of human in vivo gene expression following an exposure given measured human in vitro gene expression. We show that domain adaptation has been successfully achieved, with the rat and human in vitro data no longer being separable in the common latent space generated by the network. The network produces physiologically plausible predictions of human in vivo gene expression pattern following an exposure to a previously unseen compound. Moreover, we show the integration of the human in vitro data in the training of the domain adaptation network significantly improves the temporal accuracy of the predicted rat in vivo gene expression pattern following an exposure to a previously unseen compound. In this way, we demonstrate the improvements in prediction accuracy that can be achieved by combining data from distinct domains.

A bold vision in nanofabrication is the assembly of functionalmolecular structures using a scanning probe microscope (SPM). Thisapproach requires continuous monitoring of the molecular configurationduring manipulation. Until now, this has been impossible because theSPM tip cannot simultaneously act as an actuator and an imaging probe.Here, we implement configuration monitoring using experimental dataother than images collected during the manipulation process. We modelthe manipulation as a partially observable Markov decision process(POMDP) and approximate the actual configuration in real time usinga particle filter. To achieve this, the models underlying the POMDPare precomputed and organized in the form of a finite-state automaton,allowing the use of complex atomistic simulations. We exemplify theconfiguration monitoring process and reveal structural motifs behindmeasured force gradients. The proposed methodology marks an importantstep toward the piece-by-piece creation of supramolecular structuresin a robotic and possibly automated manner.

Lateral connections play an important role for sensory processing in visual cortex by supporting discriminable neuronal responses even to highly similar features. In the present work, we show that establishing a biologically inspired Mexican hat lateral connectivity profile along the filter domain can significantly improve the classification accuracy of a variety of lightweight convolutional neural networks without the addition of trainable network parameters. Moreover, we demonstrate that it is possible to analytically determine the stationary distribution of modulated filter activations and thereby avoid using recurrence for modeling temporal dynamics. We furthermore reveal that the Mexican hat connectivity profile has the effect of ordering filters in a sequence resembling the topographic organization of feature selectivity in early visual cortex. In an ordered filter sequence, this profile then sharpens the filters' tuning curves.

Predicting salient regions in natural images requires the detection of objects that are present in a scene. To develop robust representations for this challenging task, high-level visual features at multiple spatial scales must be extracted and augmented with contextual information. However, existing models aimed at explaining human fixation maps do not incorporate such a mechanism explicitly. Here we propose an approach based on a convolutional neural network pre-trained on a large-scale image classification task. The architecture forms an encoder–decoder structure and includes a module with multiple convolutional layers at different dilation rates to capture multi-scale features in parallel. Moreover, we combine the resulting representations with global scene information for accurately predicting visual saliency. Our model achieves competitive and consistent results across multiple evaluation metrics on two public saliency benchmarks and we demonstrate the effectiveness of the suggested approach on five datasets and selected examples. Compared to state of the art approaches, the network is based on a lightweight image classification backbone and hence presents a suitable choice for applications with limited computational resources, such as (virtual) robotic systems, to estimate human fixations across complex natural scenes. Our TensorFlow implementation is openly available at https://github.com/alexanderkroner/saliency.

In clinical trials, animal and cell line models are often used to evaluate the potential toxic effects of a novel compound or candidate drug before progressing to human trials. However, relating the results of animal and in vitro model exposures to relevant clinical outcomes in the human in vivo system still proves challenging, relying on often putative orthologs. In recent years, multiple studies have demonstrated that the repeated dose rodent bioassay, the current gold standard in the field, lacks sufficient sensitivity and specificity in predicting toxic effects of pharmaceuticals in humans. In this study, we evaluate the potential of deep learning techniques to translate the pattern of gene expression measured following an exposure in rodents to humans, circumventing the current reliance on orthologs, and also from in vitro to in vivo experimental designs. Of the applied deep learning architectures applied in this study the convolutional neural network (CNN) and a deep artificial neural network with bottleneck architecture significantly outperform classical machine learning techniques in predicting the time series of gene expression in primary human hepatocytes given a measured time series of gene expression from primary rat hepatocytes following exposure in vitro to a previously unseen compound across multiple toxicologically relevant gene sets. With a reduction in average mean absolute error across 76 genes that have been shown to be predictive for identifying carcinogenicity from 0.0172 for a random regression forest to 0.0166 for the CNN model (p < 0.05). These deep learning architecture also perform well when applied to predict time series of in vivo gene expression given measured time series of in vitro gene expression for rats.