BACKGROUND:Proper positioning during sleep is critical for musculoskeletal and neurological development of preterm neonates, yet current manual assessment methods are time-intensive and subject to inter-observer variability. METHODS:In this proof-of-concept study, we retrospectively investigated body alignment (supine, prone, right lateral, left lateral, semi-prone) and positioning of infants during sleep episodes. We used the Infant Positioning Assessment Tool (IPAT) scoring from recordings obtained during daily routine care at two NICUs. Anatomical keypoints were annotated manually and used for fine-tuning the ViTPose pose estimator and as input for machine learning classifiers to predict alignment and suboptimal positioning. RESULTS:A total of 1510 images from 50 stable preterm neonates during their quiet periods in closed incubators (gestational age 23-36 weeks) were assessed. Posture classification of five categories achieved a macro F1-score = 0.80 (95% CI: 0.74, 0.86). Suboptimal positioning prediction reached AUROC = 0.86 (0.79, 0.87) and AUPRC = 0.69 (0.59, 0.79). Input image to decision arrangement yielded AUROC = 0.87 (0.84, 0.92) and AUPRC = 0.70 (0.61, 0.82). CONCLUSIONS:Video-based automated posture and positioning assessment is feasible, enabling continuous monitoring and early warning of positioning-related complications. IMPACT:Positioning of preterm neonates is critical for musculoskeletal and neurological development; however, no automated visual assessment model is available. Visual machine learning models can assess infant positioning and predict repositioning needs. Performance saturates across architecturally diverse classifiers, indicating that further gains require improved labeling consistency rather than more complex models. Continuous 24/7 NICU incubator camera images can be used to detect positioning deterioration between routine nursing assessments.

Visual monitoring of vital parameters in premature infants has become an intensively studied area in recent years. Among these parameters, respiration rate (RR) is one of the most critical vital signs, making non-contact measurement of respiration a key research focus. Many published algorithms achieve improved performance when an appropriate region of interest (ROI) is detected prior to RR estimation. Typically, such ROIs are generated using data-driven segmentation methods. However, modern deep learning–based ROI detection algorithms require thousands of annotated samples for training, and manual data collection and annotation are time-consuming and labor-intensive. In this work, we propose a motion–periodicity–based method to automatically generate respiration-related region masks that capture the abdominal or chest area of neonates. The predicted masks were validated against independent expert annotations, achieving high localization consistency on the torso ( S_geo=0.987 ) and significant overlap with clinical ground truth (mean IoU=0.580 and Dice=0.735). We further show that these automatically produced labels can be directly used to train common segmentation architectures, eliminating the need for manual annotation and enabling the creation of large, high-quality datasets for neonatal respiration analysis. Our findings demonstrate that automatic dataset generation is both feasible and effective for training deep learning–based ROI detectors in this domain.

We introduce a novel system for automatic assessment of newborn and preterm infant behavior—including activity levels, behavioral states, and sleep–wake cycles—in clinical settings for streamlining care and minimizing healthcare professionals’ workload. While vital signs are routinely monitored, the previously mentioned assessments require labor-intensive direct observation. Research so far has already introduced non- and minimally invasive solutions. However, we developed a system that automatizes the preceding evaluations in a non-contact way using deep learning algorithms. In this work, we provide a Gated Recurrent Unit (GRU)-stack-based solution that works on a dynamic feature set generated by computer vision methods from the cameras’ video feed and patient monitor to classify the activity phases of infants adapted from the NIDCAP (Newborn Individualized Developmental Care Program) scale. We also show how pulse rate variability (PRV) data could improve the performance of the classification. The network was trained and evaluated on our own database of 108 h collected at the Neonatal Intensive Care Unit, Dept. of Neonatology of Pediatrics, Semmelweis University, Budapest, Hungary.

The objective evaluation of an infant’s activity and sleep pattern is critical in improving the comfort of the babies and ensuring the proper amount of quality sleep. The predefined behavioral states of an infant describe their consciousness and arousal level. The different states are characterized by different movements, body tone, eye movements and breath patterns. To recognize and adapt to these states is an essential part of development-friendly caring. It affects the neonate’s sleep, influencing their brain development, while improving the bonding between mother and baby, and feeding is more successful during the state of quiet awakened. It can be a more difficult task to determine the level of arousal in premature neonates. In preterm clinics, the general practice is continuous observation, requiring the attention of the hospital staff. To create an automated, more objective system, helping the hospital staff and the parents, we developed a multi-RNN (multi-recurrent neural network) network-based solution to solve this classification problem, which works on a time-series-like feature set, extracted from cameras’ video feeds. The set is composed of video actigraphy features, video-based respiration signal and additional descriptors. We separate infant caring from undisturbed presence based on our previous ensemble network solution. The network was trained and evaluated using our database of 402 h of footage, collected at the Neonatal Intensive Care Unit, Dept. of Neonatology of Pediatrics, Dept. of Obstetrics and Gynecology, Semmelweis University, Budapest, Hungary, with all-day recordings of 10 babies.

Laser Speckle Contrast Imaging (LSCI) is an optical method mainly used for creating blood flow maps. Despite its beneficial properties, the technique is yet to find a place in clinical practice. In this work, we propose a setup for LSCI to overcome some of the disadvantages associated with the method. We call the setup the sample-in-the-loop LSCI as it is based on the feedback of the captured image, which is determined by the properties of the sample and the experimental setup. We investigate and demonstrate the method in three exemplary scenarios: optimization to specific contrast setpoint, sensitivity maximization and dynamic range maximization. These goals are achieved by using optimization on the laser light pulse sequence and on the exposure time of the digital camera.

Visual monitoring of vital parameters of premature infants has become a heavily researched topic in recent years. Respiration rate (RR) is one of the most essential vital sign of newborns, therefore non-contact measurement of respiration is also a strongly studied area. Most of the published algorithms are able to provide better results if a suitable ”region of interest” (ROI) detection takes place before the estimation of RR. This ROI is typically generated with a data-driven segmentation method. However, modern deep learning-based ROI detection algorithms require several thousands of annotated samples for training. Data collection and annotation is a long and tedious process. In this work, we propose a motion periodicity based solution to automatically detect the respiration mask containing the belly or the back of neonates. The places of the automatically generated masks showed a 96% agreement with the places of the manually marked regions. We showed that by using these automatically generated respiration masks for training U-Net variants we can not just avoid the manual labeling, but also reach greater accuracy in the ultimate RR calculation. We concluded, that it is possible and worthwhile to automatically generateannotated dataset for deep learning based ROI detectors in the mentioned field.

Laser speckle contrast imaging (LSCI) is a method to visualize and quantify tissue perfusion and blood flow. A common flaw in LSCI variants is their sensitivity to the optical setup parameters and that they operate well only on statistics of undistorted laser speckle patterns. The signal saturation of the sensors makes the contrast calculation misleading; hence the illumination level must be well controlled. We describe the theoretical explanation for the saturation-caused degradation. We introduce a linear extrapolation method to eliminate the overexposure induced error up to an extent of 60-70% saturated pixel count. This, depending on the contrast value and use case, enables to use 3-8 times higher external illumination level with no deterioration of the contrast calculation and thus the measured blood flow index. Our method enables a higher signal-to-noise ratio in darker areas by allowing the use of higher illumination, utilizing a larger portion of the dynamic range of the sensors, and making the illumination level setting less cumbersome.

The temperature feedback of solid state laser diodes and various photovoltaic devices is critical for stable and reliable usage. We demonstrate that with a simple and passive electrical measurement process and optical calibration method the temperature of a photodiode can be determined, while keeping its original purpose. We present the idea with simulation and experiments, and provide an illumination-based parameter fitting method. The presented use case is a 5.6mm IR laser diode. We used its monitor photodiode to track the temperature inside the package with significantly better temporal resolution than the off-the-shelf external heat sink sensor provides. We reached a time resolution in the range of milliseconds and achieved precision of ±10mK with ±3K absolute, uncalibrated precision. The technique is not restricted to laser diodes, it is applicable to photovoltaic cells, photodiode arrays, organic diodes as well.

Laser speckle contrast imaging is a technique to determine blood flow rate with a limitation of low dynamic range. In this Letter, we introduce a varied illumination speckle contrast imaging method. It utilizes varying illumination during exposure to customize the correlation time (flow rate) to speckle contrast relation. The method can cover an order of magnitude larger range flow rate in a single exposure compared to constant illumination methods. The proposed method enables high dynamic range flow rate imaging, which is advantageous in studying larger vessels and small arteries. We demonstrate the theory by simulations and ex vivo and in vivo measurements.

The appearance of the common artifacts of laser speckle contrast imaging (LSCI), namely the granularity in flow rate estimation caused by static scatterers, is a well-known phenomenon. This artifact can be greatly reduced in spatial speckle contrast calculation using interframe decorrelated illumination, forcing true ensemble averaging. We propose a statistical model, which describes the effect of multiple image acquisitions on the contrast map quality when the illumination stable and when the illumination is decorrelated frame by frame. We investigate the improvement as a function of the ratio of dynamic and static scatterers by formulating a statistical distribution based model, using in simulation, flow phantom and in vivo experiments. Our main finding is that the ensemble averaging yields limited improvement in several practical cases due to the highly heterogeneous scatterer structure of living tissues.

Non-contact visual monitoring of vital signs in neonatology has been demonstrated by several recent studies in ideal scenarios where the baby is calm and there is no medical or parental intervention. Similar to contact monitoring methods (e.g., ECG, pulse oximeter) the camera-based solutions suffer from motion artifacts. Therefore, during care and the infants’ active periods, calculated values typically differ largely from the real ones. In this way, our main contribution to existing remote camera-based techniques is to detect and classify such situations with a high level of confidence. Our algorithms can not only evaluate quiet periods, but can also provide continuous monitoring. Altogether, our proposed algorithms can measure pulse rate, breathing rate, and to recognize situations such as medical intervention or very active subjects using only a single camera, while the system does not exceed the computational capabilities of average CPU-GPU-based hardware. The performance of the algorithms was evaluated on our database collected at the Ist Dept. of Neonatology of Pediatrics, Dept of Obstetrics and Gynecology, Semmelweis University, Budapest, Hungary.

The article describes a reference and training set free incrementally trained deep learning algorithm for camera-based respiration monitoring systems. The algorithm uses a model based discriminator to find salient areas having respiration like periodic motion. It stores the first principle component of the found waveforms into two slowly growing set along with negative, uncorrelated motion patterns. Using these samples, it trains a deep neural network classifier incrementally to recognize respiration from sudden and motion intensive situations. The classifier had no forgetting mechanism and it is able to adapt quickly the changing respiration patterns and conditions. The algorithm has been validated in a total of 24 hours diverse recording captured in the neonatal intensive care unit (NICU) of the Ist Dept. of Pediatrics and, II. Dept. of Obstetrics and Gynecology, Semmelweis University, Budapest, Hungary and in the COHFACE publicly available dataset of adult subjects. The clinical data set evaluation resulted in mean absolute error (MAE) 6.9 and root mean squared error (RMSE) of 9.8 breaths per minute, respectively, the MAE was below 5 breaths per minute for over 50% of the time. The algorithm was assessed in the COHFACE dataset of adult subjects as well with respiration estimation MAE and RMSE values of 0.95 and 1.7 breaths per minute.

Remote photoplethysmography (RPPG) is a camera-based optical technique for detecting volumetric changes of organs. This technique enables the non-contact measurement of respiration and pulse. Monitoring newborn infants is a challenging task, due to the weak pulse signals, the rather irregular respiratory pattern, and the frequent movements. Therefore, heavy optimization of the sensing and evaluation process is a must in a resource-limited embedded vision system. This is a two-faceted study, with a special focus on low computational complexity. In the field of respiration monitoring, the paper introduces an optimized convolutional neural network (CNN) and a novel, light Long Short-term Memory (LSTM) motion classifier with a narrow CNN layer. From heart rate measurement point of view, a skin segmentation based algorithm is presented. The performance of each algorithm is evaluated on a database collected at the Ist Dept. of Neonatology of Pediatrics, Dept of Obstetrics and Gynecology, Semmelweis University, Budapest, Hungary.

A camera and machine learning based system, developed at SZTAKI and by Peter Pazmany at Catholic University Budapest [L1], enables continuous non-contact measurement of respiration and pulse of premature infants. It also performs high precision monitoring, immediate apnoea warnings and logging of motion activity and caring events.

Smart laparoscope device was developed and integrated into the ROBIN HEART surgery robot system. Miniaturised silicon based force sensors were developed and integrated into laparoscope tweezers for the special applications. Different sensors were applied to detect tactile information at the tip of the laparoscope and to measure the clamping force between the tweezers. Preliminary tests were accomplished to evaluate the force and tactile signals of the integrated sensors during interventions. Tactile measurements were implemented on artificial and real animal tissues to prove the applicability of the device for biomechanical screening during Minimal Invasive Surgery.

3D force sensors were developed to further integration in laparoscopic heads of surgery robots. The Si sensors operate with piezoresistive transduction principle by measuring the stress induced signals of the symmetrically arranged four piezoresistors in a deforming membrane. As the chip size has to be reduced to a few mm(2), the conventional anisotropic alkaline etching technique was replaced by deep reactive ion etching (DRIE) for membrane formation. Moreover, DRIE enables to form any geometry of the membrane and offers the formation of monolith force transfer rod protruding over the chip surface. This rod increases shear sensitivity of the structure, thereby plays crucial role in tactile sensing. The technology applies SOI (silicon on insulator) wafers of appropriate device layer thickness, which provide highly uniform membranes and reproducibility of the process.According to the medical and functional requirements the sensors must be covered by biocompatible and sterilisable elastic polymers. As the elastomer drastically effect on the performance of the device, the proposed sensor structures were modelled by coupled finite element simulation to determine the appropriate geometric parameters meet the functional requirements. Sensors were covered with spherically shaped PDMS (polydimethylsiloxane) polymer and the effect of the elastic coating was also studied in terms of sensitivity and response time. Finally, the design of the laparoscopic head with the integrated 3D MEMS force sensors is also demonstrated.

In this work we present a complex, wireless, ambient energy powered and easy-to-use solution for vibration analysis. It is designed to incorporate the latest commercial technologies and achievements in the field of energy harvesting and wireless sensor networks with an emphasis on energy efficient spectrum estimation algorithms for embedded systems. This solution is realized on a small printed circuit board and contains all the necessary circuit components for hybrid energy harvesting; acceleration sensing; data acquisition, storing and analysis; and wireless communication. The on-board microcontroller was programmed to choose the most energy-efficient data handling algorithm (direct transfer or embedded analysis) based on the weighed combination of user settings and ambient energy. We tested and calibrated our system in laboratory environment with reference sensors, as well as in an engine room, simulating practical applications.

In this work we present a low noise, hardware efficient, and scalable read-out architecture for piezoresistive mechanical transducers containing multiple sensing elements. To reach the thermal noise limit the sensing elements are driven by modulated, differential stimuli at separated frequencies, their current are summed and digitalized for signal processing and response extraction. The solution decreases the complexity of the analog read-out electronics and makes it easily scalable. Besides the improved signal-to-noise ratio the principle can achieve minimised power consumption and self-heating of piezoresistors with minimal analogue hardware resources. The distinguishing features of the arrangement are the multiple frequency modulation, the current based multiple sensor integration, one AD converter, no analog multiplexing, and the need for only a half Wheatstone bridge per sensing element.

3D contact force sensors were developed and integrated in a demonstration system for testing the feasibility of their application in minimal invasive surgery (MIS). Piezoresistive MEMS based vectorial force sensors were designed and fabricated by 3D silicon micromachining technology and packaged according to their proposed transducer and sensor applications. In this work we demonstrated the integrability and functional applicability of the 3D force sensors in MIS robotic systems to improve their flexibility and reliability by providing real-time force and tactile information during the operation.The final goal is to integrate the new subsystems in the Robin Heart surgery robot of FRK. [1] Three different functions are targeted: 1. Micro-joystick actuator to be integrated in the hilt of the laparoscope to easily control robotic movement during operation. 2. Force sensor inside the laparoscopic jaw to provide feedback to the surgeon by measuring the grasping strength and 3.3D force/tactile sensor which facilitates palpation for tissue diagnostics during operation. In this paper we demonstrate a feasibility study regarding these proposed applications. (C) 2016 Published by Elsevier Ltd.

We propose a scalable, low-noise imager architecture for terahertz recordings that helps to build large-scale integrated arrays from any field-effect transistor (FET)- or HEMT-based terahertz detector. It enhances the signal-to-noise ratio (SNR) by inherently enabling complex sampling schemes. The distinguishing feature of the architecture is the serially connected detectors with electronically controllable photoresponse. We show that this architecture facilitate room temperature imaging by decreasing the low-noise amplifier (LNA) noise to one-sixteenth of a non-serial sensor while also reducing the number of multiplexed signals in the same proportion. The serially coupled architecture can be combined with the existing read-out circuit organizations to create high-resolution, coarse-grain sensor arrays. Besides, it adds the capability to suppress overall noise with increasing array size. The theoretical considerations are proven on a 4 by 4 detector array manufactured on 180 nm feature sized standard CMOS technology. The detector array is integrated with a low-noise AC-coupled amplifier of 40 dB gain and has a resonant peak at 460 GHz with 200 kV/W overall sensitivity.