Healthcare delivery organizations such as hospitals are complex infrastructures comprising a broad range of networked devices. They include connected medical devices which can deliver health care, support hospitals' operations, and can exchange patients' data over healthcare network protocols. Previous research has pointed out weaknesses in the implementations of some of these protocols, and demonstrated how they could be abused by malicious actors in hospitals. There are still other healthcare network protocols for which we have limited knowledge, and no security analysis can be found in the literature. This can represent an issue, as these technologies may also have vulnerabilities which could, if exploited, impact hospitals' operations and patients' data. For this reason, we investigate in this paper three healthcare protocols found in hospital networks: the POCT1-A and LIS02-A standards used by some point-of-care and laboratory devices, and the proprietary protocol Data Export used by some Philips patient monitors. We explain how to build a test lab to perform security research on medical devices, in which we demonstrate four attacks highlighting how the selected protocols can be abused. This research provides greater knowledge of threats relevant to healthcare delivery organizations, and helps to enhance network security monitoring capabilities such as intrusion detection systems. More specifically, signatures can be created to detect attacks on these protocols and datasets can be assembled to assist the development and testing of hospital-specific intrusion detection systems.

Today, there are millions of third-party Android applications. Some of them are buggy or even malicious. To identify such applications, novel frameworks for automated black-box testing and dynamic analysis are being developed by the Android community. Code coverage is one of the most common metrics for evaluating effectiveness of these frameworks. Furthermore, code coverage is used as a fitness function for guiding evolutionary and fuzzy testing techniques. However, there are no reliable tools for measuring fine-grained code coverage in black-box Android app testing. We present the Android Code coVerage Tool, ACVTool for short, that instruments Android apps and measures code coverage in the black-box setting at class, method and instruction granularity. ACVTool has successfully instrumented 96.9% of apps in our experiments. It introduces a negligible instrumentation time overhead, and its runtime overhead is acceptable for automated testing tools. We demonstrate practical value of ACVTool in a large-scale experiment with Sapienz, a state-of-the-art automated testing tool. Using ACVTool on the same cohort of apps, we have compared different coverage granularities applied by Sapienz in terms of the found amount of crashes. Our results show that none of the applied coverage granularities clearly outperforms others in this aspect.

Current approaches to estimate the risk of compromise are based on either historical data or pure technical assessments, such as the number and severity of vulnerabilities in the target network. We propose a novel experimental approach for estimating the risk of compromise based on experimental data, as opposed to observational data, by leveraging on cyber ranges and capture the flag exercises. We identify the key design principles in terms of response and explanatory variables, specification of how they can be measured, and the overall block design from related experiments and approaches as well as assess their suitability and limitations.

Cryptojacking applications pose a serious threat to mobile devices. Due to the extensive computations, they deplete the battery fast and can even damage the device. In this work we make a step towards combating this threat. We collected and manually verified a large dataset of Android mining apps. In this paper, we analyze the gathered miners and identify how they work, what are the most popular libraries and APIs used to facilitate their development, and what static features are typical for this class of applications. Further, we analyzed our dataset using VirusTotal. The majority of our samples is considered malicious by at least one VirusTotal scanner, but 16 apps are not detected by any engine; and at least 5 apks were not seen previously by the service. Mining code could be obfuscated or fetched at runtime, and there are many confusing miner-related apps that actually do not mine. Thus, static features alone are not sufficient for miner detection. We have collected a feature set of dynamic metrics both for miners and unrelated benign apps, and built a machine learning-based tool for dynamic detection. Our BrenntDroid tool is able to detect miners with 95% of accuracy on our dataset.

Web applications are the target of many well known exploits and also a fertile ground for the discovery of security vulnerabilities. Yet, the success of an exploit depends both on the vulnerability in the application source code and the environment in which the application is deployed and run. As execution environments are complex (application servers, databases and other supporting applications), we need to have a reliable framework to test whether known exploits can be reproduced in different settings, better understand their effects, and facilitate the discovery of new vulnerabilities. In this paper, we present TestREx - a framework that allows for highly automated, easily repeatable exploit testing in a variety of contexts, so that a security tester may quickly and efficiently perform large-scale experiments with vulnerability exploits. It supports packing and running applications with their environments, injecting exploits, monitoring their success, and generating security reports. We also provide a corpus of example applications, taken from related works or implemented by us.

Free and Open Source Software (FOSS) components are ubiquitous in both proprietary and open source applications. Each time a vulnerability is disclosed in a FOSS component, a software vendor using this component in an application must decide whether to update the FOSS component, patch the application itself, or just do nothing as the vulnerability is not applicable to the older version of the FOSS component used. This is particularly challenging for enterprise software vendors that consume thousands of FOSS components and offer more than a decade of support and security fixes for their applications. Moreover, customers expect vendors to react quickly on disclosed vulnerabilities—in case of widely discussed vulnerabilities such as Heartbleed, within hours. To address this challenge, we propose a screening test: a novel, automatic method based on thin slicing, for estimating quickly whether a given vulnerability is present in a consumed FOSS component by looking across its entire repository. We show that our screening test scales to large open source projects (e.g., Apache Tomcat, Spring Framework, Jenkins) that are routinely used by large software vendors, scanning thousands of commits and hundred thousands lines of code in a matter of minutes. Further, we provide insights on the empirical probability that, on the above mentioned projects, a potentially vulnerable component might not actually be vulnerable after all.

The work presented in this paper is motivated by the need to estimate the security effort of maintaining Free and Open Source Software (FOSS) components within the software supply chain of a large international software vendor. We investigated publicly available factors (from number of active users to commits, from code size to usage of popular programming languages, etc.) to identify which ones impact three potential effort models: centralized (the company checks each component and propagates changes to the product groups), distributed (each product group is in charge of evaluating and fixing its consumed FOSS components), and hybrid (seldom used components are checked individually by each development team, the rest is centralized). We use Grounded Theory to extract the factors from a six months study at the vendor. We report the results on a sample of 166 FOSS components used by the vendor.

Code coverage is an important metric that is used by automated Android testing and security analysis tools to guide the exploration of applications and to assess efficacy. Yet, there are many different variants of this metric and there is no agreement within the Android community on which are the best to work with. In this paper, we report on our preliminary study using the state-of-the-art automated test design tool Sapienz. Our results suggest a viable hypothesis that combining different granularities of code coverage metrics can be beneficial for achieving better results in automated testing of Android applications.

The deluge of Android apps from third-party developers calls for sophisticated security testing and analysis techniques to inspect suspicious apps without accessing their source code. Code coverage is an important metric used in these techniques to evaluate their effectiveness, and even as a fitness function to help achieving better results in evolutionary and fuzzy approaches. Yet, so far there are no reliable tools for measuring fine-grained bytecode coverage of Android apps. In this work we present ACVTool that instruments Android apps and measures the smali code coverage at the level of classes, methods, and instructions. Tool repository: https://github.com/pilgun/acvtool

Extracting knowledge from data using data analytical techniques requires identifying the research goal, collecting data, preparing the data, exploring the data, developing analytical models, and analyzing the developed models. The knowledge could be acquired by testing theoretical models using empirical research methods such as data analytics. Testing theoretical models using data analytical techniques requires expressing the problem to be investigated as a question, which may be divided into subquestions. The main challenge in data analytics is the availability of data. Often, the researcher needs to collect artifacts that could be used for testing theoretical models. Data collection is iterative. At the beginning, the researcher may start with an initial dataset that has a limited set of attributes. Such dataset may be provided by the entity that has an interest in the research results or was identified by the researcher. Collected datasets have often data attributes where their values are frequently missing.

Background: Static analysis security testing (SAST) tools may be evaluated using synthetic micro benchmarks and benchmarks based on real-world software. Aims: The aim of this study is to address the limitations of the existing SAST tool benchmarks: lack of vulnerability realism, uncertain ground truth, and large amount of findings not related to analyzed vulnerability. Method: We propose Delta-Bench - a novel approach for the automatic construction of benchmarks for SAST tools based on differencing vulnerable and fixed versions in Free and Open Source (FOSS) repositories. To test our approach, we used 7 state of the art SAST tools against 70 revisions of four major versions of Apache Tomcat spanning 62 distinct Common Vulnerabilities and Exposures (CVE) fixes and vulnerable files totalling over 100K lines of code as the source of ground truth vulnerabilities. Results: Our experiment allows us to draw interesting conclusions (e.g., tools perform differently due to the selected benchmark). Conclusions: Delta-Bench allows SAST tools to be automatically evaluated on the real-world historical vulnerabilities using only the findings that a tool produced for the analysed vulnerability.

The work presented in this paper is motivated by the need to estimate the security effort of consuming Free and Open Source Software (FOSS) components within a proprietary software supply chain of a large European software vendor. To this extent we have identified three different cost models: centralized (the company checks each component and propagates changes to the different product groups), distributed (each product group is in charge of evaluating and fixing its consumed FOSS components), and hybrid (only the least used components are checked individually by each development team). We investigated publicly available factors (e. g., development activity such as commits, code size, or fraction of code size in different programming languages) to identify which one has the major impact on the security effort of using a FOSS component in a larger software product.

Vulnerability data sources are used by academics to build models, and by industry and government to assess compliance. Errors in such data sources therefore not only are threats to validity in scientific studies, but also might cause organizations, which rely on retro versions of software, to lose compliance. In this work, we propose an automated method to determine the code evidence for the presence of vulnerabilities in retro software versions. The method scans the code base of each retro version of software for the code evidence to determine whether a retro version is vulnerable or not. It identifies the lines of code that were changed to fix vulnerabilities. If an earlier version contains these deleted lines, it is highly likely that this version is vulnerable. To show the scalability of the method we performed a large scale experiments on Chrome and Firefox (spanning 7,236 vulnerable files and approximately 9,800 vulnerabilities) on the National Vulnerability Database (NVD). The elimination of spurious vulnerability claims (e.g. entries to a vulnerability database such as NVD) found by our method may change the conclusions of studies on the prevalence of foundational vulnerabilities.

Web applications are the target of many known exploits and also a fertile ground for the discovery of security vulnerabilities. Those applications may be exploitable not only because of the vulnerabilities in their source code, but also because of the environments on which they are deployed and run. Execution environments usually consist of application servers, databases and other supporting applications. In order to test whether known exploits can be reproduced in different settings, better understand their effects and facilitate the discovery of new vulnerabilities, we need to have a reliable testbed. In this paper, we present TESTREX, a testbed for repeatable exploits, which has as main features: packing and running applications with their environments; injecting exploits and monitoring their success; and generating security reports. We also provide a corpus of example applications, taken from related works or implemented by us.