Accurate Point Cloud Registration (PCR) is an important task in 3D data processing, involving the estimation of a rigid transformation between two point clouds. While deep-learning methods have addressed key limitations of traditional non-learning approaches, such as sensitivity to noise, outliers, occlusion, and initialization, they are developed and evaluated on clean, dense, synthetic datasets (limiting their generalizability to real-world industrial scenarios). This paper introduces R3PM-Net, a lightweight, global-aware, object-level point matching network designed to bridge this gap by prioritizing both generalizability and real-time efficiency. To support this transition, two datasets, Sioux-Cranfield and Sioux-Scans, are proposed. They provide an evaluation ground for registering imperfect photogrammetric and event-camera scans to digital CAD models, and have been made publicly available. Extensive experiments demonstrate that R3PM-Net achieves competitive accuracy with unmatched speed. On ModelNet40, it reaches a perfect fitness score of 1 and inlier RMSE of 0.029 cm in only 0.007s, approximately 7 times faster than the state-of-the-art method RegTR. This performance carries over to the Sioux-Cranfield dataset, maintaining a fitness of 1 and inlier RMSE of 0.030 cm with similarly low latency. Furthermore, on the highly challenging Sioux-Scans dataset, R3PM-Net successfully resolves edge cases in under 50 ms. These results confirm that R3PM-Net offers a robust, high-speed solution for critical industrial applications, where precision and real-time performance are indispensable. The code and datasets are available at https://github.com/YasiiKB/R3PM-Net.

Memory bandwidth contention in multi-core systems severely impacts application performance and quality-of-service (QoS) guarantees. Regulating the shared memory bandwidth mitigates the memory performance uncertainty thereby making it a manageable resource and improving trustworthiness of multi-core systems. In this work we propose a memory bandwidth regulation mechanism LMS-AR, i.e., LMS Prediction-based Adaptive Regulator within a Linux kernel module to distribute the memory bandwidth as a resource among the CPU cores. We describe a design in which both monitoring and regulation is enforced from outside by a master core - which is not a dedicated controller for regulation. This allows for plugging in computationally heavy prediction and regulation algorithms without interfering with the regulated core. An adaptive filtering technique was employed for prediction of per-core bandwidth requirement. We conducted several experiments with SPEC CPU 2017 benchmarks distributed across multiple cores. Our proposed approach demonstrated significant improvement over Memguard with respect to slowdown ratios caused due to memory contention. Our solution is hosted publicly at $\href{https://github.com/ss22ongithub/LMSAdaptiveRegulator}{https://github.com/ss22ongithub/LMSAdaptiveRegulator}$.

In industrial applications such as semiconductor manufacturing, high-precision, high-speed positioning is critical for maintaining assembly throughput. These systems rely on fast and accurate sensor feedback, often achieved by fusing heterogeneous data to enable precise motion control. In our earlier work, we demonstrated multi-rate sensor fusion, combining rapid but less precise linear encoders with slower yet highly accurate vision-based detection. However, vision algorithms are computationally demanding, often exceeding available sampling periods and degrading closed-loop control performance.We address these challenges using the predictable multi-processor CompSOC platform, which ensures deterministic execution and supports parallelization of vision algorithms. We systematically analyze trade-offs by exploring two design knobs: (i) vision parallelization, which reduces the sampling period while requiring additional hardware resources, and (ii) vision subsampling and range reduction, which reduce the sampling period but decrease vision sensor precision. Furthermore, we evaluate the effect of semiconductor die orientation on quality-of-control (QoC).By incorporating these analyses into Pareto trade-offs between control performance and computational resources, our approach provides practical insights for system designers. It helps optimize hardware resources and sensor strategies for high-precision, resource-constrained industrial applications, thereby achieving high QoC. We demonstrate this on a multi-sensor positioning system implemented on a predictable multi-processor platform in a semiconductor motion stage case study.

Manufacturing processes increasingly depend on advanced production machinery that must deliver high quality and large volumes. This applies to die-bonding machines as well that, especially at the time being after the years of shortage, need to meet very high standards of speed and accuracy. To achieve this, these devices are exploring the use of computer vision algorithms for automatic recognition of wafer positioning and die size. Nevertheless, these systems are typically managed by software-only solutions, which may fall short under stringent execution time requirements. A promising solution is the use of heterogeneous platforms, combining general-purpose processors with reconfigurable hardware. Such platforms offer the flexibility to handle both software tasks, which benefit from operating system support, and critical functions requiring hardware acceleration. This article presents a closed-loop implementation of a vision-based multisensor control system for an industrial application. The implementation exploits the capabilities of system on module technologies to provide flexible input/output and software execution coupled with computing acceleration for the vision algorithm on the reconfigurable field-programmable gate array (FPGA) fabric. The FPGA coprocessor has been designed leveraging the high-level synthesis technology and optimized on a dataset of 10 k realistic images to meet the industrial use case's performance, communication, and accuracy requirements. Moreover, the resulting accelerator performance and resource utilization demonstrate the possibility of reaching state-of-the-art metrics of handwritten hardware designs while allowing for higher abstraction and productivity of the design process.

Robotic and automotive platforms increasingly rely on electric motors, demanding sophisticated control strategies. Handing over the optimization process for critical objectives such as disturbance rejection and energy efficiency to autonomous systems can significantly streamline the design effort. In this work, we propose an actor-critic reinforcement learning framework designed to optimize both speed regulation and energy efficiency through its reward function, while explicitly embedding hardware constraints into the training loss function. Specifically, we introduce a novel formulation that encodes violations of physical control constraints directly into the actor loss function, thereby integrating application-specific hardware limitations into the learning process. We demonstrate the effectiveness of this approach by comparing it against classical reward-shaping based methods. Our results show significant improvements in constraint handling and compliance during training and operational deployment. This study addresses key challenges associated with deploying model-free neural network-based control strategies, contributing to enhanced safety and reliability in reinforcement learning-derived control policies.

Digital Twins (DTs) are virtual representations of physical systems or Physical Twins (PTs) that are used for various data-driven applications in manufacturing industry such as predictive maintenance, diagnostics and condition monitoring. The data in DTs is collected through virtual sensors, which augment the physical sensors by providing additional data that cannot be directly observed. In high-throughput production systems like semiconductor manufacturing equipment, highspeed and high-precision motion stages control the equipment's movement. However, the use of DTs in these systems is limited due to their real-time requirements. To enable real-time applications, the DT must be synchronized with its physical counterpart to ensure timely data from virtual sensors. The synchronization mechanism should be non-blocking to prevent any impact on the throughput of physical systems. In this paper, we propose a synchronization mechanism for DTs in high-speed high-precision motion control systems. The mechanism involves sharing PT states with DT over a network and compensating for network delays. The synchronization mechanism is validated in a framework comprising an industrial motion stage system and its digital twin. The validation is done for different synchronization delay scenarios, demonstrating its effectiveness of proposed approach in eliminating synchronization delays without blocking the PT operation. The proposed mechanism enables real-time virtual sensing ensuring data timeliness with high accuracy.

Many autonomous systems are increasingly adopting Neural Networks (NNs) based perception in vision-in-the-loop (VIL) control systems. In many industrial applications, the features (shape, size and texture) of the object of interest varies, which imposes robustness requirements on the perception algorithm. Further, performance of the VIL system imposes strict latency requirements. Using NNs in VIL system poses two challenges. First, the NN models should be lightweight resulting in a low closed-loop latency. Second, availability of representative training data for ensuring robustness of the lightweight NN models. Collecting such training data is expensive and often, infeasible in many industrial systems. In this work we propose an approach for training the NNs used for VIL applications using digital twins (DT). The DT is used for automatically generating and labelling training data representing various features like object shapes and directional lighting. Starting from a lightweight NN base model, our proposed approach fine-tunes or retrains the model using DT-generated training data achieving desired performance and robustness on a different target operating condition. The approach is validated considering a VIL semiconductor motion stage system with square and rectangular dies of dimension of (0.5cm × 0.5cm) and (0.5cm × +1cm) respectively. The VIL system limits the positioning error in the range of 2% compared to 12% positioning error with no vision feedback.

In recent literature, it has been shown that the number of steps in a sequential quadratic programming algorithm for a non-linear model predictive control (NMPC) problem can be greatly reduced by a parallel shooting method. The efficiency of such a parallel shooting method further depends on how the algorithm is implemented on parallel computing platforms such as Graphics Processing Units (GPUs). The GPU implementation should consider the degree of parallelism necessary for higher time efficiency as well as the hardware resource consumption/limitation at the GPU for a given problem size. In this paper, we present a multilevel parallel GPU implementation for sequential quadratic programming and an (Alternating Direction Method of Multipliers) ADMM solver. First, we introduce a GPU implementation enabling parallel computing of many quadratic programs (QPs) by functional parallelism. Next, we parallelize each QP solver using data parallelism of basic linear matrix operations. We show that the proposed GPU implementation greatly scales with the degree of parallelism in the parallel shooting method. Further, we show how a GPU implementation can be configured for a given problem size avoiding resource overprovisioning.

The high-speed, high-precision positioning of objects is a critical component in various industrial manufacturing processes. The semiconductor die packaging, for instance, requires the precise pickup and placement of semiconductor dies on substrates. This is done by coupling the silicon wafer which contains thousands of semiconductor dies, with a motion control platform equipped with linear motor and encoder. The motion controller relies on linear motor and encoder to accurately position the silicon wafer at reference positions, which are determined through the relative positions of the dies on the wafer. However, the challenge arises when neighboring dies get misaligned during the pickup process, making it impossible to read the position of the die through encoder. This paper addresses the challenge of precise alignment in high-speed, micro-scale manufacturing environments, where traditional methods struggle due to the disconnect between the point-of-interest (dies) and point-of-control (motor/silicon wafer). To overcome these challenges, we propose a Deep Neural Network (DNN) based perception that allows for robust sensing of die positions. We also propose a fusion mechanism to incorporate this vision feedback with the encoder to accurately detect the misalignment and compensate for it before periodic pickups of the dies. We use a software-in-the-loop validation framework to demonstrate that our proposed method could successfully eliminate the misalignment before the pickup in the range under consideration.

Robotic and automotive platforms are rapidly expanding in features and are incorporating more and more electric motor components. Consequently, the energy efficiency of motor control systems emerges as a major design challenge. The process of formulating and fine-tuning specialized speed regulation strategies for each application becomes progressively more laborious and expensive. A reinforcement learning agent specialized in electrical motor dynamics, capable of generalizing across a wide range of possible end-use applications, presents a promising and convenient solution. In this article, we introduce a novel design of a reinforcement learning agent, grounded in time series analysis, intended for application-agnostic electric motor control that optimizes both speed regulation and energy efficiency. Trained on the motor's internal dynamics, the agent provides operating point-specific control inputs, eliminating the need for manual tuning and application system-identification. Compared to application tuned classical control methods, the agent exhibited on-par or improved speed regulation performance and demonstrated advanced capability to save energy, showcasing its potential for future applications.

This paper explores the challenges and methodologies for analyzing the timing and performance of safety-critical real-time robotics and control applications on Multiprocessor System-on-Chip (MPSoC) platforms, i.e., CompSOC in this study. Leveraging a predictable Publish-Subscribe communication protocol, implemented based on a wait-free FIFO channel, we focus on the Worst Case Response Time (WCRT) analysis of interdependent hard real-time tasks in a processing chain, scheduled by a Timed Division Multiplexing (TDM) scheduler. For this purpose, after we define the execution model of Publish-Subscribe based real-time applications on CompSOC, we propose two distinctive WCRT analysis approaches: a deterministic calculation approach, and a stochastic approach based on Timed Automata (TA) theory and UPPAAL Stochastic Model Checker (UPPAAL-SMC) for a tighter WCRT bound. Moreover, building upon the stochastic TA model, we propose an approach for performance analysis, particularly focusing on factors such as message loss-rate and sensor-to-actuator throughput to present the percentage of the sample data that passes all the stages in the processing chain and affect the system behaviour. Our experimental validations, conducted on an implemented instance of the CompSOC MPSoC platform on the programmable logic (FPGA) of a Pynq board, affirm the effectiveness of our approach.

The high-precision and high-speed positioning systems require position feedback with high accuracy at a higher frequency. As reported in recent literature, high accuracy and high operating frequency can be achieved by fusing multiple position sensor data, e.g., the linear encoder (less robust/accurate, but fast) and object detection using camera images (accurate, but slow due to heavy processing load). Typically, image-based object detection incurs a significant computational delay due to computationally intensive processes and is the main performance bottleneck. Moreover, the computation delay varies when implemented on industrial platforms and degrades the performance of the closed-loop control system. In this paper, we present scheduling techniques such as parallelism and pipelining considering predictable multi-core platforms for such a multi-sensor positioning system. On the one hand, the predictable platform nearly removes the variation in execution time, making the delay constant. On the other hand, the parallel and pipeline schedules reduce the computation delay, translating to a shorter sampling period and better closed-loop performance. Furthermore, we perform a design space exploration on various parameters and control performance considering an industrial case study of semiconductor die-bonding equipment.

In these years of severe chip shortage, it is even more important to improve the efficiency of chip manufacturing. As is well known, manufacturing phases rely on increasingly intelligent production machinery, which must ensure high quality and large volumes. That is true also for die-bonding machines, which are required to satisfy very high standards of speed and accuracy. For this purpose, such devices have started to evaluate the possibility of adopting computer vision algorithms for automatic recognition of wafer positioning and die size. This paper proposes an FPGA accelerated implementation of one of these algorithms, demonstrating the advantages of using this technology to this end and paving the way towards a larger adoption of this kind of acceleration platform for the different tasks composing modern industrial motion control systems. At the state of the art, such systems are typically managed with software-oriented solutions, which may not be sufficient in the case of highly restrictive requirements in terms of execution time. For this reason, the design flow considered the use of high-level hardware design, which offers a more software-friendly solution to developers without in-depth hardware knowledge. The proposed solution is a state-of-the-art implementation for execution time and resources of prograimnable logic, while enabling higher precision in terms of die position estimation.

Modern safety-critical cyber-physical systems such as medical imaging equipment or autonomous vehicles need to respect strict deadlines on received data-processing workloads. These deadlines and workloads are dynamic and uncontrollable and the systems typically have only a limited discrete number of system configurations to respond to dynamic changes. The number and types of processors allocated to a data-processing task, their operating voltage and frequency, and the resolution and frequency of sensing (e.g., images) are examples of controllable configuration parameters. Guaranteeing dynamically changing deadlines under uncontrollable workloads with a limited discrete number of response options can be phrased as a multi-objective tracking problem for a switched max-plus linear system. This results in a combined scheduling and control problem. We propose an integrated state-feedback and model-predictive control solution that minimizes the number of deadline misses and the cost of implementation (e.g., energy consumption). We demonstrate the effectiveness of our approach through simulation.

Accurate object positioning is critical in many industrial manufacturing applications. The execution time and precision of the object positioning task have a significant impact on the overall performance and throughput, especially in cost-sensitive industries such as semiconductor manufacturing. In addition, the object positioning algorithm must adapt to changes in object size, features, and environmental conditions in real-time. While traditional sensors struggle to cope with dynamic conditions, vision-based perception is more adaptable and robust. Vision-based perception can capture and analyze visual information by using cameras and image processing algorithms, providing a robust way to locate objects in dynamic environments. However, classical perception algorithms based on vision cannot handle objects with different characteristics, and modern object detectors that rely on deep neural networks struggle to adapt to image sizes, resulting in unnecessary computations. To address these challenges, this paper proposes an approach for designing a branched multi-input deep neural network (DNN) that considers variations in input image sizes to adapt the input branches. In essence, the proposed DNN reduces the computation time for images with lower dimensions. To validate the proposed approach, an IC dataset is created that represents the variations in object sizes as seen in semiconductor manufacturing machines. Depending on the choice of input branches, the average inference time is reduced by over 30% with a slight gain in detection accuracy.

With the growing demand for semiconductor products, the semiconductor manufacturing industries are trying to increase their production capacities. Additional requirements and constraints are also enforced on semiconductor manufacturing equipment, particularly on robustness for visual inspections and vision-based alignment. Deep neural networks (DNNs) are prominently used for vision-based tasks to improve robustness. The challenge, however, is that semiconductor manufacturing industries still use brownfield systems and equipment with legacy hardware and software. The legacy systems introduce challenging requirements and constraints on the DNN deployment and the traditional approach to inference optimization results in poor inference performance. This paper presents a structured approach to optimize the inference of DNNs for vision-based tasks for industrial brownfield architectures with existing legacy hardware, software, and the associated requirements and constraints. Four directions in the machine learning operations (MLOps) pipeline are explored in this approach - DNN architecture selection, DNN model optimization, target deployment platform, and inference engine - while adhering to the legacy systems’ requirements and constraints. We present our approach using the case study from the semiconductor manufacturing industry that deploys DNNs for vision-based position detection in their legacy equipment. The results of the optimized DNN deployment are compared with a baseline implementation, and up to 44% improvement in inference timing performance is achieved without compromising on inference accuracy.

In this paper, we propose a parallel shooting algorithm for solving nonlinear model predictive control problems using sequential quadratic programming. This algorithm is built on a two-phase approach where we first test and assess sequential convergence over many initial trajectories in parallel. However, if none converge, the algorithm starts varying the Newton step size in parallel instead. Through this parallel shooting approach, it is expected that the number of iterations to converge to an optimal solution can be decreased. Furthermore, the algorithm can be further expanded and accelerated by implementing it on GPUs. We illustrate the effectiveness of the proposed Parallel Shooting Sequential Quadratic Programming (PS-SQP) method in some benchmark examples for nonlinear model predictive control. The developed PS-SQP parallel solver converges faster on average and especially when significant nonlinear behaviour is excited in the NMPC horizon.

Industrial motion control systems, e. pick-andplace tasks in semiconductor manufacturing equipment, require precise positioning for achiering high machine throughput. Linear encoders are the standard industrial sensors used for position feedback due to their relatively low cost, high resolution, and high operating frequency. The challenge is that the linear encoders measure the positions at the points-of-control of the equipment, eg motors, and not at the points-of-interest, e.g. pick-and-place positions The coupling between a point-of-control and the point-of-interest is affected by external disturbances such as mechanical misalignment of the product, friction, and warping of the material, and linear encoders fail to sense these disturbances. Vision-based sensing is a potential alternative to achieve robust sensing and high-precision control. However, vision processing has a long computational delay and affects the machine throughput. In this paper, we propose a multi-rate multi-sensor fusion approach to improve the positioning accuracy of industrial motion control systems with different points-of-control and pointsof-interest. We present a multi-rate Kalman filter with bias correction to fuse accurate but slow and delayed vision sensor data with fast but less accurate linear encoder data for highprecision position control. We validate the proposed method in an evaluation framework by considering an industrial case study of a semiconductor die-bonding machine. $\mathbf{A}$ design-space exploration is done to evaluate the performance of the proposed solution with respect to various relevant design parameters. The effectiveness of the proposed solution depends on the type of disturbances and vision processing delay. For the parameter range under consideration, we achieve a positioning accuracy of $1 \mu \mathrm{m}$.

Vehicle platooning is an enabler technology for increasing road capacity, improving safety and reducing fuel consumption. Platoon control is a two-layered system where each layer runs under a different communication standard and rate – (i) the upper-layer operates under a specific V2V communication standard such as IEEE 802.11p and (ii) the lower-layer operates over high-speed in-vehicle communication networks such as FlexRay, CAN. The upper-layer, under 802.11p, uses periodic Cooperative Awareness Messages (CAMs) for exchanging vehicle motion information (i.e., acceleration, velocity and so on), the rate of which is adapted depending on the network congestion level. With over 70% channel load, the CAMs experience significant delay and packet loss, jeopardizing the stability of the platoon control. Under such high congestion, the European Telecommunications Standard Institute (ETSI) proposes to engage Decentralized Congestion Control (DCC) to control the channel load. We propose a platoon control and DCC scheme to tackle this scenario. Our contribution is three-fold. First, we propose a multi-layer platoon model explicitly augmenting the communication delay in the state-space. Second, the augmented delay-aware platoon model is integrated in the state-of-the-art multi-layer multi-rate model predictive control (MPC) for the upper-layer. Third, we adopt a message-rate congestion control scheme to keep the channel load under a given threshold. We use the proposed delay-aware MPC scheme under the message-rate congestion control scheme which may lead to switching under dynamic network conditions. Using the proposed technique, we show that platoon performance can be maintained under high network congestion while maintaining string stability.

Industrial applications and processes such as quality inspections, pick and place operations, and semiconductor manufacturing require accurate positioning control for achieving the high throughput of the assembly machines. Vision-based sensing is considered to be a potential means to achieve robust positioning control which is referred to as a vision-in-the-loop (VIL) system. In such motion systems, the point-of-control and the point-of-interest are often different due to several physical factors. In this case, validation of a system is done only when a machine prototype is available. A physical prototype is often expensive and infeasible in real-life. This paper proposes an evaluation framework for VIL systems targeting a predictable multi-core embedded platform. The presented framework offers model-in-the-loop (MIL), software-in-the-loop (SIL), and processor-in-the-loop (PIL) simulation features for evaluating the closed-loop performance of industrial motion control systems. As a deployment platform, we consider a predictable embedded platform CompSOC. The predictable nature of the CompSOC platform guarantees periodic and deterministic execution of the control applications and allows verification of the timing properties and performance of the VIL system. Additionally, the framework offers automatic code generation feature targeting the CompSOC platform. Closed-loop simulation setup models the system dynamics and camera position in the CoppeliaSim physics simulation engine and simulates the system software in C and MATLAB. CoppeliaSim runs as a server and MATLAB as a client in synchronous mode. We show the effectiveness of our framework using a vision-based motion control example.