Extended finite state machines (EFSMs) model stateful systems with internal data variables and have numerous applications in software engineering. A major advantage of this type of model lies in its ability to model both the data flow and the data-dependent control behaviour. In the absence of such models, it is desirable to reverse-engineer them by observing the system's behaviour. However, existing approaches generally require the ability to reset the system during inference, or can only handle situations where the control flow depends exclusively on the input parameters, and not on the values of the stored data. In this work, we present a black-box active learning algorithm that infers EFSMs with guards and registers, and which significantly relaxes the assumptions that have to be made about the system in comparison to previous attempts.

This paper presents an active inference method for Extended Finite State Machines, where inputs and outputs are parametrized, and transitions can be conditioned by guards involving input parameters and internal variables called registers. The method applies to (software) systems that cannot be reset, so it learns an EFSM model of the system on a single trace.

In recent years, the significance of test logs in ensuring system reliability and diagnosing runtime events has grown significantly, particularly with software expanding into various domains, necessitating rigorous verification and validation processes. However, the complexity and cost of testing have prompted a shift towards automation. This paper addresses the challenges of automated software testing through root-cause event detection. The proposed approach initially involves parsing and partitioning logs, followed by representing test events as dense vectors in a continuous space, enabling the capture of semantic similarities and relationships among events based on their sequence positions. Subsequently, test events are clustered in this embedded space, and each log partition is represented as a vector, with its characteristics reflecting the number of events in the log partition present in the clusters. Through two distinct case studies, we demonstrate that the final clustering of log partitions in this new space efficiently identifies root cause events. We evaluate our approach on two applications and anticipate its contribution as a cornerstone for future research and deployment of automated log mining.

Models that can represent the behavior of systems, such as a Finite State Machine (FSM), are crucial for software development and maintenance as they serve as a base for several automated activities like testing, verification, validation, and refinement of systems. Contrasting their importance and value, models are usually complex and costly to obtain. Model inference algorithms can help with this task. In this paper, we propose a method to improve the learning process of FSMs by inferring separating sequences from traces and using them in characterization sets. We conducted a case study to assess the impact of the proposed method on an FSM learning algorithm called hW-inference. We observed that the proposed method was capable of improving by 24% the learning process.

Extended finite state machines (EFSMs) model stateful systems with internal data variables, and have many software engineering applications, including system analysis and test case generation. Where such models are not available, it is desirable to reverse engineer them by observing system behaviour, but existing approaches are either limited to classical FSM models with no internal data state, or implicitly require the ability to reset the system under inference, which may not always be possible. In this paper, we present an extension to the hW-inference algorithm that can infer EFSM models, complete with guards and internal data update functions, from systems without a reliable reset, although there are currently some restrictions on the type of system and model.

Extended finite state machines (EFSMs) model stateful systems with internal data variables, and have many software engineering applications. It is possible to infer such models by observing system behaviour. Still, existing approaches are either limited to classical FSM models with no internal data state, or implicitly require the ability to reset the system under inference, which may not always be possible. We present an extension to the hW-inference algorithm that can infer EFSM models, with input and output parameters as well as guards and internal registers and their data update functions, from systems without a reliable reset. For the problem to be tractable, we require some assumptions on the observability and determinism of the system. The main restriction is that the control flow of the system must be finite, although data types could be infinite.

Model inference is a form of systematic testing of black-box systems while learning at the same time a model of their behaviour. In this paper, we study the impact of W-set reduction in hW-inference, an inference algorithm for learning models from scratch. hW-inference relies on progressively extending a sequence h into a homing sequence for the system, and a set W of separating sequences into a fully characterizing set. Like most other inference algorithms, it elaborates intermediate conjectures which can be refined through counterexamples provided by an oracle. We observed that the size of the W-set could vary by an order of magnitude when using random counterexamples. Consequently, the length of the test suite is hugely impacted by the size variation of the W-set. Whereas the original hW-inference algorithm keeps increasing the W-set until it is characterizing, we propose reassessing the set and pruning it based on intermediate conjectures. This can lead to a shorter test suite to thoroughly learn a model. We assess the impact of reduction methods on a self-scanning system as used in supermarkets, where the model we get is a finite state machine with 121 states and over 1800 transitions, leading to an order of magnitude of around a million events for the trace length of the inference.

Telemetry data (e.g.: CPU and memory usage) is an essential source of information for a software system that projects the system’s health. Anomalies in telemetry data warn system administrators about an imminent failure or deterioration of service quality. However, input events to the system (such as service requests) are the cause of abnormal system behaviour and, thus, anomalous telemetry data. By observing input events, one might predict anomalies even before they appear in telemetry data, thus giving the system administrator even earlier warning before the failure. Finding a correlation between input events and anomalies in telemetry data is challenging in many cases. This paper proposes a machine learning approach to learn the causality correlation between input event sequences and telemetry data. To this aim, a Natural Language Processing(NLP) approach is employed to create a concept space model to distinguish between normal and abnormal test sequences. Based on a vectorized representation of each input sequence, the concept space indicates whether the sequence will cause a system failure. Since the meaning of fault is not established in system status Telemetry-based fault detection, the suggested technique first detects periods of time when a software system status encounters aberrant situations (Bug-Zones). An extensive study on a real-world database acquired by a telecommunication operator and an open-source microservice software demonstrates that our approach achieves 71% and 90% accuracy as a Bug-Zones predictor.

Automated fault identification in long test logs is a tough problem, mainly because of their sequential character and the impossibility of constructing training sets for zero-day faults. To reduce software testers' workload, rule-based approaches have been extensively investigated as solutions for efficiently finding and predicting the fault. Based on software system status monitoring log analysis, we propose a new learning-based technique to automate anomaly detection, correlate test events to anomalies and predict system failures. Since the meaning of fault is not established in system status monitoring-based fault detection, the suggested technique first detects periods of time when a software system status encounters aberrant situations (Bug-Zones). The suggested technique is then tested in a real-time system for anomaly prediction of new tests. The model may be used in two ways. It can assist testers to focus on faulty-like time intervals by reducing the number of test logs. It may also be used to forecast a Bug-Zone in an online system, allowing system administrators to anticipate or even prevent a system failure. An extensive study on a real-world database acquired by a telecommunication operator demonstrates that our approach achieves 71 % accuracy as a Bug-Zones predictor.

Continuous integration in software development requires to run the tests on a regular basis to ensure that the code does not regress. So that the execution time of the regression test suite remains reasonable its size must be reduced while preserving its fault detection capability. From test execution logs, we extract from each test a trace, which is a sequence of events. We then consider each event as a word, and apply natural language processing methods, here the Word2Vec tool, to detect similarities in sentences, and partition them into clusters. We thus can reduce the regression test suite by removing redundant tests. We present the approach on a small case study, and we use mutation based testing to assess the effectiveness of the reduction.

Directed fuzzing focuses on automatically testing specific parts of the code by taking advantage of additional information such as (partial) bug stack trace, patches or risky operations. Key applications include bug reproduction, patch testing and static analysis report verification. Although directed fuzzing has received a lot of attention recently, hard-to-detect vulnerabilities such as Use-After-Free (UAF) are still not well addressed, especially at the binary level. We propose UAFuzz, the first (binary-level) directed greybox fuzzer dedicated to UAF bugs. The technique features a fuzzing engine tailored to UAF specifics, a lightweight code instrumentation and an efficient bug triage step. Experimental evaluation for bug reproduction on real cases demonstrates that UAFuzz significantly outperforms state-of-the-art directed fuzzers in terms of fault detection rate, time to exposure and bug triaging. UAFuzz has also been proven effective in patch testing, leading to the discovery of 30 new bugs (7 CVEs) in programs such as Perl, GPAC and GNU Patch. Finally, we provide to the community a large fuzzing benchmark dedicated to UAF, built on both real codes and real bugs.

We present an efficient approach to retrieve behavioural models from reactive software systems in the form of Finite State Machines by testing them. The system is accessed in black box mode; thus, no source or binary code is needed. The novelty of the approach is that it does not require to reset the system between tests (queries) and does not require any knowledge of the system apart from its input domain. Experiments have shown that it can scale up to systems that may have thousands of states. (C) 2019 Elsevier Inc. All rights reserved.

This paper assesses a method to reverse engineer models from software or cyber-physical systems that cannot be reset. This is done through active learning, and can compare favourably as a systematic testing method.

The paper focuses on the problems of passive and active FSM inference as well as checking sequence generation. We consider the setting where an FSM cannot be reset so that its inference is constrained to a single trace either given a priori in a passive inference scenario or to be constructed in an active inference scenario or aiming at obtaining checking sequence for a given FSM. In each of the last two cases, the expected result is a trace representing a checking sequence for an inferred machine, if it was not given. We demonstrate that this can be achieved by a repetitive use of a procedure that infers an FSM from a given trace (identifying a minimal machine consistent with a trace) avoiding equivalent conjectures. We thus show that FSM inference and checking sequence construction are two sides of the same coin. Following an existing approach of constructing conjectures by SAT solving, we elaborate first such a procedure and then based on it the methods for obtaining checking sequence for a given FSM and inferring a machine from a black box. The novelty of our approach is that it does not use any state identification facilities. We demonstrate that the proposed approach can also be constrained to find a solution in a subset of FSMs represented by a nondeterministic mutation machine. Experiments with a prototype implementation of the developed approach using an existing SAT solver indicate that it scales for FSMs with up to a dozen of states and requires relatively short sequences to identify a black box machine.

Industrial Control Systems are found often in industrial sectors and critical infrastructures to monitor and control industrial processes. Recently, the security of industrial control systems has gained much attention as these systems now exhibit an increased interaction with the Internet. In fact, classical SCADA systems are already lacking with security problems, and with the increased interconnectivity to the Internet, they are now exposed to new types of threats and cyber-attacks. Intrusion detection technology is one of the most important security solutions used today in industrial control systems to detect potential attacks and malicious activities. This paper summarizes previous work for Intrusion Detection Systems approaches in Industrial Control Systems and highlights challenges and opportunities in implementing such solutions. We believe that such insights are valuable for further research in the industrial security context.

This article proposes a temporal and parametric specification language (PARTRAP) developed for the verification of execution traces. The language extends specification patterns with nested scopes, real-time and first-order quantification over the data inside a JSON trace, while remaining pragmatic. Its design was directed by a case study in the medical field (computer aided surgery). The paper briefly presents the case study and details the design rationale, syntax and semantics of the language. The language has been implemented and several properties have been successfully evaluated over a corpus of 100 surgery traces.

Efficient detection and instrumentation of function calls is fundamental for a variety of dynamic analysis techniques, including dynamic callgraph construction, control-flow integrity, and automatic vulnerability discovery. A common way of detecting calls at the machine code level is to look for CALL instructions. However, optimizing compilers frequently implement function tail calls with JMP instructions instead, and distinguishing an intra-procedural jump from a JMP-based function call is not straightforward. Despite the importance of making this distinction, prior research has not produced a reliable solution. In this paper, we address the problem of dynamic function call detection in real-time. We propose a heuristic-based approach named iCi to efficiently and automatically instrument calls, including conventional CALLs and JMP-based calls, at runtime. iCi does not rely on source code, debug information, symbol tables or static analysis. We show that iCi achieves an f-score of 0.95 in the worst case, regardless of optimization level. We open-source our implementation as well as the oracle we used for our evaluation.1

Machine learning in the form of inference of state machine models has gained popularity in model-based testing as a means of retrieving models from software systems. By combining an old idea from machine inference with methods from automata testing in a heuristic approach, we propose a new promising direction for inferring black box systems that cannot be reset. Preliminary experiments show that this heuristic approach scales up well and outperforms more systematic approaches.