The safe and secure implementation of increasingly complex features is a major challenge in the development of autonomous and distributed embedded systems. Automated design-time procedures that guarantee the fulfillment of critical system properties are a promising approach to tackle this challenge. In the European project XANDAR, which took place from 2021 to 2023, eight partners developed an X-by-Construction (XbC) design framework to support developers in the creation of embedded software systems with certain safety, security, and real-time properties. The design framework combines a model-based toolchain with a hypervisor-based runtime architecture. It targets modern high-performance hardware, facilitates the integration of machine learning applications, and employs a library of trusted safety and security patterns to reduce the implementation and verification effort. This paper describes the concepts developed during the project, the prototypical implementation of the design framework, and its application in both an automotive and an avionics use case.

The follow-up of the development of the premature baby is a major component of its clinical care since it has been shown that it can reveal a pathology. However, no method allowing an automated and continuous monitoring of this development has been proposed. Within the framework of the Digi-NewB European project, our team wishes to offer new clinical indices qualifying the maturation of newborns. In this study, we propose a new method to characterize motor activity from video recordings. For this purpose, we have chosen to characterize the motion temporal organization by drawing inspiration from sleep organization. Thus, we propose a fully automatic process allowing to extract motion features and to combine them to estimate a functional age. By investigating two datasets, one of 28.5 hours (manually annotated) from 33 newborns and one of 4,920 hours from 46 newborns, we show that the proposed approach is relevant for monitoring in clinical routine and that the extracted features reflect the maturation of preterm newborns. Indeed, a compact and interpretable model using gestational age and three motion features (mean duration of intervals with motion, total percentage of time spent in motion and number of intervals without motion) was designed to predict post-menstrual age of newborns and showed an admissible mean absolute error of 1.3 weeks. While the temporal organization of motion was not studied clinically due to a lack of technological means, these results open the door to new developments, new investigations and new knowledge on the evolution of motion in newborns.

Automotive and aviation systems are undergoing a radical shift in their software and hardware architectures, affecting the processes and communities used to design them. On a technical level, we see a trend towards integration of heterogeneous function domains on centralized computing platforms. On a process and collaboration level, this trend implies two things: First, heterogeneous communities of OEMs and suppliers on different tiers need to collaborate intensely to create innovative software-intensive products. Second, these communities need to be able to exchange development artifacts efficiently by means of open, model-based exchange formats. Even competing companies will have to collaborate in such heterogeneous communities. We illustrate the challenges of trustful, model-based information exchange in heterogeneous development communities that arise due to intellectual property protection concerns. We identify data security threats for collaborative, model-based engineering processes and suggest guidelines that support trustful information exchange between partners of a heterogeneous community.

The integration of connected and autonomous technologies in safety-critical brought significant system design challenges. These systems are constantly evolving and becoming more complex. With their connection to the cloud and the internet, these safety-critical systems are now exposed to greater risks of cyber-attacks, which poses new challenges to their safety, reliability and resilience. To approach these complex system design challenges, this paper proposes XANDAR’s Verification & Validation strategy using Static Analysis, Timing Analysis, Model-in-loop and Network simulation tool. To ensure functional correctness, the proposed XANDAR Verification and Validation approach utilizes early integration of simulation and static analysis techniques during the development cycle. This proposed approach differs from existing methods by emphasizing early integration, rather than applying it to later stages of development cycle to begin verification. In addition, the proposed approach utilizes timing analysis to ensure non-functional timing aspects meet the timing requirements. The approach applies tools such as Polyspace Bug Finder and Code Prover for static analysis, Timing Architect for timing analysis, NS3 simulator for network architecture simulation. The proposed approach aims to ensure system safety and security through a rigorous and comprehensive verification process. These verification approaches will be validated by applying it to automotive and avionics use cases.

The adherent need for computational power in highly automated driving will lead to fewer control units with larger computational power, eventually replacing a large number of a vehicle’s electric control units by so-called vehicle control computers. As a result, high-level functionality will no longer be realized by co-designing software and hardware, but instead by integrating software from different suppliers onto a shared hardware platform. In order to ensure that this functionality will operate according to system specifications, the integrator will need to ensure that requirements related to performance and timing are fulfilled. However, solely relying on e.g. periodical or event triggered activation patterns when integrating third-party software without taking into account external triggering information may lead to overly pessimistic estimations, thus leading to unnecessarily expensive hardware or even preventing feasible configurations. In this work, we provide a comprehensive overview of activation patterns that are employed in the automotive industry and a detailed description of their semantics. We also propose novel event models corresponding to these activation patterns in order to enable the application of performance analysis techniques. We demonstrate the usage of these models on a case study in the context of compositional performance analysis, and quantify the improvement when bounding response times in comparison to the usage of chained task sets. Our experimental results show that the accurate representation of occurrence schemes using specialized event models allows to reduce the pessimism by up to 32.3% compared to using traditional modeling techniques. Finally, we discuss the results along with the impact on the determinism of an application’s temporal behavior.

The integration of connected and autonomous technologies in safety-critical and cyber-physical systems offers great potential in the vital application domains of transportation, manufacturing and aerospace. These technological advancements are necessary to meet the increasing demand for intelligent services, as they open doors to new business models by analysing and sharing the generated data. However, where this sharing of mix-critical data and broader connectivity brings opportunities, it simultaneously presents serious cybersecurity and safety risks due to the cyber-physical nature of these systems. Hence, delivering these intelligent services securely, safely, and reliably to its consumers is a complex engineering and design problem. One of the ways to approach this engineering problem is to consider both system functional and non-functional properties (safety, security, reliability) and systematically integrate them across system design and operational life cycle. The XANDAR project investigates this approach and aims to develop holistic software design methods and architectures for safety-critical and cyber-physical systems that guarantee functional and non-functional properties “byconstruction”. This paper focuses on the non-functional aspects of the project and discusses the preliminary work. by presenting the core cybersecurity principles and uses them as a baseline to propose a holistic cybersecurity engineering process. The tasks of the proposed cybersecurity engineering process are also map onto relevant clauses of ISO 21434. In future, proposed work will be integrated into the XANDAR software toolchain and validated for an avionics situation perception pilot assistance and automotive autonomous driving use cases.

Software-centered development processes take a more and more prominent place in automotive system design. Accommodating the growing complexities resulting from the increasing heterogeneity in automotive hardware, software, and their collaborative integration requires new workflows. To address this challenge, we propose an approach for system decomposition based on a behavior description integrated with an architecture description language. Additionally, we consider timing validations as a crosscutting concern during different stages of the development and describe an automation concept to support a correct-by-construction development process. Initial user feedback indicates that our concepts together with a proper tool support will help engineers during system design and speed up the process.

Realizing desired properties "by construction" is a highly appealing goal in the design of safety-critical embedded systems. As verification and validation tasks in this domain are often both challenging and time-consuming, the by-construction paradigm is a promising solution to increase design productivity and reduce design errors. In the XANDAR project, partners from industry and academia develop a toolchain that will advance current development processes by employing a model-based X-by-Construction (XbC) approach. XANDAR defines a development process, metamodel extensions, a library of safety and security patterns, and investigates many further techniques for design automation, verification, and validation. The developed toolchain will use a hypervisor-based platform, targeting future centralized, AI-capable high-performance embedded processing systems. It is co-developed and validated in both an avionics use case for situation perception and pilot assistance as well as an automotive use case for autonomous driving.

A newborn is preterm if birth occurred before a gestational age of 37 weeks. He has several immature functions, which implies a specific monitoring and, among others, the analysis of its sleep. Here we make a focus on Quiet Sleep (QS), whose increasing is primordial with age, and characterized by an absence of motion and a regular cardiorespiratory rhythm. A method to automatically detect QS is proposed, on the basis of a video analysis (detection of motion), supplemented by the estimation of ECG and respiration “qualities”. This approach combines feature extraction and machine learning methods. It was validated on a set of 15 newborns and 25 eight-hours recordings manually annotated. Best results were obtained by combining non-motion intervals and ECG quality, but showing also an overestimation of $QS (Se=88\%,\ Sp=49\%)$. However, regarding extracted features, we observed similar trends between manual and automated QS, with an increasing of average duration of QS intervals and percentage of time in QS with age, also approaching values of the full-term newborns. Finally, computation of QS on a larger set of 45 recordings confirmed the interest of the approach for maturation evaluation purposes.

The next generation of networked embedded systems (ES) necessitates rapid prototyping and high performance while maintaining key qualities like trustworthiness and safety. However, development of safety-critical ES suffers from complex software (SW) toolchains and engineering processes. Moreover, the current trend in autonomous systems, which relies on Machine Learning (ML) and AI applications when combined with fail-operational requirements renders the Verification and Validation (V&V) of these new systems a challenging endeavor. Prime examples are Advanced Driver-Assistance Systems (ADAS) that are prone to various safety/security vulnerabilities. The XANDAR project aims at developing a mature SW toolchain (from requirements analysis to the actual code integration on target including V&V) fulfilling the needs of industry for rapid prototyping of interoperable and autonomous ES. Starting from a model-based system architecture, XANDAR will leverage automatic model synthesis and software parallelization techniques to achieve specific non-functional requirements setting the foundation for a novel (real-time, safety-, and security)-by-Construction paradigm.

Video-based motion analysis recently appeared to be a promising approach in neonatal intensive care units for monitoring the state of preterm newborns since it is contact-less and noninvasive. However it is important to remove periods when the newborn is absent or an adult is present from the analysis. In this paper, we propose a method for automatic detection of preterm newborn presence in incubator and open bed. We learn a specific model for each bed type as the camera placement differs a lot and the encountered situations are different between both. We break the problem down into two binary classifications based on deep transfer learning that are fused afterwards: newborn presence detection on the one hand and adult presence detection on the other hand. Moreover, we adopt a strategy of decision intervals fusion in order to take advantage of temporal consistency. We test three deep neural network that were pre-trained on ImageNet: VGG16, MobileNetV2 and InceptionV3. Two classifiers are compared: support vector machine and a small neural network. Our experiments are conducted on a database of 120 newborns. The whole method is evaluated on a subset of 25 newborns including 66 days of video recordings. In incubator, we reach a balanced accuracy of 86%. In open bed, the performance is lower because of a much wider variety of situations whereas less data are available.

Background: Sleep is an important determinant of brain development in preterm infants. Its temporal organization varies with gestational age (GA) and post-menstrual age (PMA) but little is known about how sleep develops in very preterm infants. The objective was to study the correlation between the temporal organization of quiet sleep (QS) and maturation in premature infants without severe complications during their neonatal hospitalization. Methods: Percentage of time spent in QS and average duration of time intervals (ADI) spent in QS were analyzed from a cohort of newborns with no severe complications included in the Digi-NewB prospective, multicentric, observational study in 2017-19. Three groups were analyzed according to GA: Group 1 (27-30 weeks), Group 2 (33-37 weeks), Group 3 (>39 weeks). Two 8-h video recordings were acquired in groups 1 and 2: after birth (T1) and before discharge from hospital (T2). The annotation of the QS phases was performed by analyzing video recordings together with heart rate and respiratory traces thanks to a dedicated software tool of visualization and annotation of multimodal long-time recordings, with a double expert reading. Results are expressed as median (interquartile range, IQR). Correlations were analyzed using a linear mixed model. Results: Five newborns were studied in each group (160 h of recording). Median time spent in QS increased from 13.0% [IQR: 13-20] to 28.8% [IQR: 27-30] and from 17.0% [IQR: 15-21] to 29.6% [IQR: 29.5-31.5] in Group 1 and 2, respectively. Median ADI increased from 54 [IQR: 53-54] to 288 s [IQR: 279-428] and from 90 [IQR: 84-96] to 258 s [IQR: 168-312] in Group 1 and 2. Both groups reach values similar to that of group 3, respectively 28.2% [IQR: 24.5-31.3] and 270 s [IQR: 210-402]. The correlation between PMA and time spent in QS or ADI were, respectively 0.73 (p < 10-4) and 0.46 (p = 0.06). Multilinear analysis using temporal organization of QS gave an accurate estimate of PMA (r 2 = 0.87, p < 0.001). Conclusion: The temporal organization of QS is correlated with PMA in newborns without severe complication. An automated standardized continuous behavioral quantification of QS could be interesting to monitor during the hospitalization stay in neonatal units.

Preterm newborns are prone to late-onset sepsis, leading to a high risk of mortality. Video-based analysis of motion is a promising non-invasive approach because the behavior of the newborn is related to his physiological state. But it is needed to analyze only images where the newborn is solely present in incubator. In this context, we propose a method for video-based detection of newborn presence. We use deep transfer learning: bottleneck features are extracted from a pre-trained deep neural network and then a classifier is trained with these features on our database. Moreover, we propose a strategy that allows to take advantage of temporal consistency. On a database of 11 newborns with 56 days of video recordings, the results show a balanced accuracy of 80%.

In this paper we present work towards automating two process steps supporting the optimization of the runnable-to-task mapping in automotive multi-core control units. We describe these steps in close relation to the AUTOSAR methodology to facilitate the integration with existing design processes. The first step is the automated generation of an initial configuration that balances the core utilization using constraint programming. The second step is the optimization of an existing configuration based on dynamic system behavior using an evolutionary algorithm. An abstract intermediate representation provides interoperability with existing AUTOSAR tools. We use a small case study to evaluate the feasibility of our approach.

A crucial step for developing and testing a system of facial expression analysis is to choose the database which suits best the targeted context application. We propose in this paper a survey based on the review of 69 databases, taking into account both macro- and micro-expressions. To the best of our knowledge, there are no other surveys with so many databases. We review the existing facial expression databases according to 18 characteristics grouped in 6 categories (population, modalities, data acquisition hardware, experimental conditions, experimental protocol and annotations). These characteristics are meant to be helpful for researchers when they are choosing a database which suits their context application. We bring to light the trends between posed, spontaneous and in-the-wild databases, as well as micro-expression databases. We finish with future directions, including crowd sourcing and databases with groups of people.

The amount of safety-critical embedded systems in automotive development is heavily growing. Ensuring their reliability not only increases the complexity of functions but also requires determinism at design and execution time, which is considerably challenging to fulfill and verify for multi-core processors. The Logical Execution Time (LET) is recently recognized in automotive industry as an approach for ensuring deterministic functional behavior. However, to decrease the manual design effort and time for deploying such complex systems to multi-core platforms and for ensuring their strict timing and safety requirements, automatic solutions are needed. This work presents a solution for allocating tasks to multi-core processors and generating a time-triggered schedule for embedded systems considering safety, timing, and LET semantics. The approach we propose solves both challenges by defining them as a Constraint Satisfaction Problem (CSP). To examine our CSP formulation, we use MiniZinc, which is a solver-independent constraint modeling language that can employ a variety of solvers. In a case study, we explore optimizations of an industrial system that are enabled by scheduling and task allocation design decisions. Further, the performance of the proposed solutions is evaluated based on large set of synthetically generated system models.

In order to analyze expressions that are different from the prototypic expressions defined by Ekman, manifold learning has been proposed to build person-specific continuous representations of facial expressions. Yet, it is still a challenging problem to build such a manifold with no prior knowledge on the morphology of the subject. Here, we propose a method to build a person-specific manifold of facial expressions able to adapt to the morphology of the subject in an unsupervised manner. The manifold is initialized with the facial landmarks of the neutral face and 5 synthesized basic expressions. Our first contribution is to detect automatically the neutral face of the subject so that we can build the manifold in an unsupervised manner. Our second and main contribution is to adapt in an unsupervised manner the initialized manifold to the morphology of the subject by detecting the real basic expressions of the subject while maintaining constraints in the manifold. Our third contribution is to perform the adaptation on spontaneous expressions with typical head pose variation for human-computer interaction. The experiments show that the adaptation works well on posed expressions and that the constraints for the adaptation on spontaneous expressions is efficient when head pose variation is considered.

Facial expression databases are essential to develop and test a system of facial expressions analysis. We propose in this paper a survey based on the review of 61 databases. To the best of our knowledge, there are no other surveys with so many databases. We identify 18 characteristics to describe the database and group them in 6 categories, (population, modalities, data acquisition hardware, experimental conditions, experimental protocol and annotations). These characteristics are useful to create or choose a database relevant to the targeted context application. We propose to classify the databases according to these characteristics so it can be helpful for researchers to choose the database suited to their context application. We bring to light the trends between posed, spontaneous and in-the-wild databases. We finish with future directions, including crowd sourcing and databases with groups of people.

The automatic analysis of emotion remains a challenging task in unconstrained experimental conditions. In this paper, we present our contribution to the 6th Audio/Visual Emotion Challenge (AVEC 2016), which aims at predicting the continuous emotional dimensions of arousal and valence. First, we propose to improve the performance of the multi-modal prediction with low-level features by adding high-level geometry-based features, namely head pose and expression signature. The head pose is estimated by fitting a reference 3D mesh to the 2D facial landmarks. The expression signature is the projection of the facial landmarks in an unsupervised person-specific model. Second, we propose to fuse the unimodal predictions trained on each training subject before performing the multimodal fusion. The results show that our high-level features improve the performance of the multi-modal prediction of arousal and that the subjects fusion works well in unimodal prediction but generalizes poorly in multimodal prediction, particularly on valence.