Effective cybersecurity testing relies on accurate threat identification to guide test design and risk mitigation. Threat modelling plays a central role in this process by helping analysts anticipate potential vulnerabilities. However, traditional threat modelling is a manual, time-consuming task that requires significant expertise, which can limit its scalability and integration into modern testing workflows.This study investigates the use of large language models (LLMs) to support and partially automate threat modelling, aiming to improve both the efficiency and coverage of cybersecurity testing. Using the STRIDE framework, we evaluate two workflows: a single-agent approach and a two-agent collaboration. We apply three LLMso1, o3, and Sonnetto a curated dataset comprising 24 system descriptions and 745 known threats.The results show that LLMs can accelerate the generation of structured threat models and identify plausible threats, including some not explicitly listed in the validation data. While LLM outputs still lack the depth and reliability of expert-created models, their use can help testers identify key risks earlier and focus test efforts more effectively.These findings suggest that LLMs can augment the threat modelling process as part of cybersecurity testing, reducing analyst workload and enhancing the overall security assurance process.
In this paper, we address the task of identifying false positive discrepancies in critical Instrumentation and Control (I&C) software testing as an Irregular Multivariate Time Series Classification (IMTSC) problem and evaluate the performance of different Neural Network architectures, including GRU-D, Transformer, and the Set Function for Time Series (SEFT). Architectures are trained on a real-world dataset from industrial test reports, and evaluated in our use case. Our finding is that the Transformer-based method outperforms the other baselines in terms of precision, while SEFT shows noteworthy performance compared to the literature benchmarks and in the lowest training time, highlighting the importance of our context-focused evaluation.
Static code analysis conducted by means of learning-based methods is an essential part of Security Testing. Effective learning algorithms are crucial for training reliable models that can accurately detect weaknesses and vulnerabilities. During models’ training, however, it is also of paramount importance to use adequate datasets of vulnerable and non-vulnerable source code. Most existing learning-based methods have been evaluated by applying them to public datasets of code fragments labeled as vulnerable and non-vulnerable. However, it is recognized that such datasets contain spurious entries, and are often imbalanced, i.e., contain a large portion of non-vulnerable code. While the first issue is often fixed with a pre-processing of data cleaning operations, the second one is almost ignored. This paper reports a preliminary study that investigates the effect of adopting imbalanced datasets and imbalance techniques on the performance of learning-based vulnerability detection methods. Our results show that (i) resampling approaches, in particular, a combination of over and under sampling, can generate reliable models and corroborate the results; and (ii) imbalance loss functions can improve the performance in case of very imbalanced and variegated datasets.
Passive testing is an approach to verify system behavior by observing logs from normal operation, without actively injecting test stimuli. This paper presents an industrial case study of applying passive testing in the domain of vehicular embedded systems, utilizing two specialized tools: Timed Easy Approach to Requirements Syntax (T-EARS) for specifying temporal requirements, and Napkin Studio for evaluating these requirements against real system execution logs.We collaborated with Volvo Construction Equipment (VCE) to translate a set of natural language requirements into structured T-EARS specifications. Then, we used Napkin Studio to test these requirements against recorded machine log data passively. We evaluate the feasibility of this approach, the extent to which it can detect requirement violations or injected faults, and the perceptions of industry stakeholders regarding the adoption of such passive tests in their verification process.The results show that a majority of functional requirements can be expressed as Guarded Assertions (GAs) and validated on logs, uncovering specific issues. Stakeholders found the method promising for improving test coverage and efficiency, although integration challenges (e.g., log signal inconsistencies and tool usability issues) were noted.Overall, this work provides empirical evidence that passive testing with T-EARS and Napkin Studio can complement traditional hardware-in-the-loop testing, offering a scalable and non-intrusive verification approach in developing vehicular systems
The paper is devoted to an efficient distinguishing sequence (DS) generation. DSs are widely used in model-based testing against Finite State Machines (FSMs), however the problem of their existence check is PSPACE-complete. Homing sequences (HSs), on the contrary, are usually shorter than DSs, and their existence can be checked faster. We therefore, study the cases when a DS can be substituted with an HS, and discuss whenever such substitution brings a shorter test case. The approach is promising under the white box testing assumption, i.e., when all potential implementations are enumerated explicitly.
