
Self-play reinforcement learning has recently emerged as a way to train driving policies without any human data. It uses cheap, large-scale simulations to substitute expensive, large-scale human driving demonstrations. A key limitation of this approach is that policies trained through pure self-play can learn effective but alien driving conventions incompatible with people. Previous works attempt to mitigate such behavioral misalignments through extensive reward engineering and domain randomization, which are brittle and labor-intensive. Instead of completely discarding human demonstrations, our method treats them as a regularization objective on top of a minimal safe goal-reaching reward. Like the spice in a good stew, we find that a little human data goes a long way: our method uses only 30 minutes of human demonstrations, 2500x fewer than comparable imitation learning approaches. Resulting policies coordinate with held-out human trajectories and complete training in 15 hours on a single consumer-grade GPU. Videos and full source code are available at https://spiced-self-play.com/.
Simulating complex unsteady physical phenomena relies on detailed mathematical models, simulated for instance by using the Finite Element Method (FEM). However, these models often exhibit discrepancies from the reality due to unmodeled effects or simplifying assumptions. We refer to this gap as the ignorance model. While purely data-driven approaches attempt to learn full system behavior from measurements, they require large amounts of high-quality data across the entire spatial and temporal domain. In many real-world scenarios, such information is unavailable as physical quantities are measurable only at limited points, making full data-driven modeling unreliable. To overcome this limitation, a hybrid twin approach is required. Instead of simulating phenomena from scratch, here we focus on the modeling of the ignorance component. Since physics-based models approximate the overall behavior of the phenomena, the remaining ignorance is typically lower in complexity than the full physical response, therefore, it can be learned with significantly fewer data. A key difficulty, however, is that spatial measurements are sparse, and even more, obtaining data measuring the same phenomenon for different spatial configurations — system’s domain, external load, etc. — is challenging in practice. Our contribution is to overcome this limitation by using Graph Neural Networks (GNNs) to represent the ignorance model. GNNs inherently handle geometric structures, making them compatible with any approximation technique, for instance, FEM. They learn the spatial pattern of the missing physics even when the number of measurement locations is limited. This allows us to enrich the physics-based model with data-driven corrections without requiring dense spatial, temporal and parametric data. To assess the performance of the proposed methodology, we evaluate this GNN-based hybrid twin on nonlinear heat transfer problems across different meshes, geometries, and load positions. The results indicate that the GNN captures model discrepancies and generalizes corrections across varying domain geometries and mesh discretizations. This leads to improvements in simulation accuracy and interpretability, while also reducing data requirements.
This paper reviews the current applications of artificial intelligence (AI) and machine learning (ML) techniques in industrial manufacturing processes. It provides a comprehensive overview of the three main learning paradigms, supervised, unsupervised, and reinforcement learning and maps their algorithmic families to four major process domains: forming, machining, joining, and additive manufacturing. For each domain, sensor modalities, data characteristics, data acquisition feasibility, and control challenges are analyzed to assess the conditions influencing ML adoption. Differences in maturity levels are shown to arise primarily from variations in data availability, sensor integration complexity, and process automation readiness. The review offers an application-oriented synthesis covering predictive maintenance, real-time process optimization, and automated quality inspection. A publication-weighted maturity matrix demonstrates that supervised learning is well established in machining and additive manufacturing, where standardized sensors and abundant labeled data enable mature AI solutions. By contrast, unsupervised methods are commonly applied for anomaly detection in domains with limited labeled data, such as joining. Reinforcement learning is emerging as a viable approach for closed-loop control, particularly in contexts where reliable digital twins and robust simulation environments support model training. Cross-cutting techniques including physics-informed learning, explainable AI, multi-fidelity surrogate modeling, and edge-to-cloud computing architectures are also discussed. The paper concludes with a proposed future research agenda emphasizing the development of open benchmark datasets, improved simulator fidelity, and interpretable hybrid models, with the broader goal of bridging the gap between academic research and industrial deployment.
In 2020, Ravi et al. [23] published three shuffling variants with each offering a different performance-security trade-off for protecting the Numeric theoretic Transform (NTT). Among them, the fine-shuffling was proposed as the lightweight variant. The idea is to randomise the order of loading and storing the operands of the butterfly computation using conditional swapping based on random control bit. However, as noted by the authors themselves, basic-fine-shuffling implementation suffered from an obvious attack on the conditional swapping mask. So they introduced the bitwise-fine-shuffling to fix this issue. In this paper, we break this implementation using a template attack. The idea is to aggregate the leakage from the 16 bitwise AND operations used to construct the swap mask. The attack has been performed both on simulation and on a target. Then, we propose a masked fine-shuffling variant to protect the fine-shuffling operation. The idea is to mask the secret-dependent memory accesses and apply the swap mask over its boolean sharing instead. The implementation is proven secure against first-order attacks in the probing model. The implementation has been benchmarked on an processor and incurs a total overhead of 25% on the entire - decapsulation algorithm, compared with 51% overhead for the defeated bitwise-fine-shuffling of [23].