There is a lack of scientific testing of commercially available malware detectors, especially those that boast accurate classification of never-before-seen (i.e., zero-day) files using machine learning (ML). Consequently, efficacy of malware detectors is opaque, inhibiting end users from making informed decisions and researchers from targeting gaps in current detectors. In this article, we present a scientific evaluation of four prominent commercial malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously and never-before-seen files? Is purchasing a network-level malware detector worth the cost? To investigate, we tested each tool against 3,536 total files (2,554 or 72% malicious and 982 or 28% benign) of a variety of file types, including hundreds of malicious zero-days, polyglots, and APT-style files, delivered on multiple protocols. We present statistical results on detection time and accuracy, consider complementary analysis (using multiple tools together), and provide two novel applications of the recent cost–benefit evaluation procedure of Iannacone and Bridges. Although the ML-based tools are more effective at detecting zero-day files and executables, the signature-based tool might still be an overall better option. Both network-based tools provide substantial (simulated) savings when paired with either host tool, yet both show poor detection rates on protocols other than HTTP or SMTP. Our results show that all four tools have near-perfect precision but alarmingly low recall, especially on file types other than executables and office files: Thirty-seven percent of malware, including all polyglot files, were undetected. Priorities for researchers and takeaways for end users are given. Code for future use of the cost model is provided.

This work presents an evaluation of six prominent commercial endpoint malware detectors, a network malware detector, and a file-conviction algorithm from a cyber technology vendor. The evaluation was administered as the first of the Artificial I ntelligence Applications t o Autonomous Cybersecurity (AI ATAC) prize challenges, funded by / completed in service of the US Navy. The experiment employed 100K files (50/50% benign/malicious) with a stratified distribution of file types, including ~1K zero-day program executables (increasing experiment size two orders of magnitude over previous work). We present an evaluation process of delivering a file to a fresh virtual machine donning the detection technology, waiting 90s to allow static detection, then executing the file and waiting another period for dynamic detection; this allows greater fidelity in the observational data than previous experiments, in particular, resource and time-to-detection statistics. To execute all 800K trials (100K files × 8 tools), a software framework is designed to choreograph the experiment into an automated, time-synced, and reproducible workflow with substantial parallelization. Software with base classes for this framework are provided. A cost-benefit model was configured to integrate the tools’ detection statistics into a comparable quantity by simulating costs of use. This provides a ranking methodology for cyber competitions and a lens for reasoning about the varied statistical results. The results provide insights on state of commercial malware detection.

To better secure critical infrastructure, especially power systems, this paper introduces a virtual SCADA automation controller. The automation controller is a gateway into a power subsystem, making it a valuable target for cyber-attacks that could cut it off from the control center and cause a loss of view and control. To prevent this, the Virtual Automation Controller (VAC) is a backup device that mirrors the capabilities of the physical controller. It can communicate via Modbus and DNP3 and is containerized so it can be deployed on a variety of platforms. Furthermore, it utilizes software-defined networking to quickly disconnect a failed automation controller and preserve its state for forensics. The VAC gives system operators time to replace the failed controller and prevents dangerous and costly damage to power systems. The VAC is compared against the SEL 3505-3 RTAC and shown to have the necessary features to act as a failover controller.

BGP route leaks frequently precipitate serious disruptions to interdomain routing. These incidents have plagued the Internet for decades while deployment and usability issues cripple efforts to mitigate the problem. Peerlock, introduced in 2016, addresses route leaks with a new approach. Peerlock enables filtering agreements between transit providers to protect their own networks without the need for broad cooperation or a trust infrastructure. We outline the Peerlock system and one variant, Peerlock-lite, and conduct live Internet experiments to measure their deployment on the control plane. Our measurements find evidence for significant Peerlock protection between Tier 1 networks in the peering clique, where 48% of potential Peerlock filters are deployed, and reveal that many other networks also deploy filters against Tier 1 leaks. To guide further deployment, we also quantify Peerlock's impact on route leaks both at currently observed levels and under hypothetical future deployment scenarios via BGP simulation. These experiments reveal present Peerlock deployment restricts Tier 1 leak export to 10% or fewer networks for 40% of simulated leaks. Strategic additional Peerlock-lite deployment at all large ISPs (fewer than 1% of all networks), in tandem with Peerlock within the peering clique as deployed, completely mitigates 80% of simulated Tier 1 route leaks.

Gartner, a large research and advisory company, anticipates that by 2024 80% of security operation centers (SOCs) will use machine learning (ML) based solutions to enhance their operations. 11 https://www.ciodive.com/news/how-data-science-tools-can-lighten-the-load-for-cybersecurity-teams/572209/ In light of such widespread adoption, it is vital for the research community to identify and address usability concerns. This work presents the results of the first in situ usability assessment of ML-based tools. With the support of the US Navy, we leveraged the national cyber range-a large, air-gapped cyber testbed equipped with state-of-the-art network and user emulation capabilities-to study six US Naval SOC analysts' usage of two tools. Our analysis identified several serious usability issues, including multiple violations of established usability heuristics for user interface design. We also discovered that analysts lacked a clear mental model of how these tools generate scores, resulting in mistrust a and/or misuse of the tools themselves. Surprisingly, we found no correlation between analysts' level of education or years of experience and their performance with either tool, suggesting that other factors such as prior background knowledge or personality play a significant role in ML-based tool usage. Our findings demonstrate that ML-based security tool vendors must put a renewed focus on working with analysts, both experienced and inexperienced, to ensure that their systems are usable and useful in real-world security operations settings.

The security of the Internet's routing infrastructure has underpinned much of the past two decades of distributed systems security research. However, the converse is increasingly true. Routing and path decisions are now important for the security properties of systems built on top of the Internet. In particular, BGP poisoning leverages the de facto routing protocol between Autonomous Systems (ASes) to maneuver the return paths of upstream networks onto previously unusable, new paths. These new paths can be used to avoid congestion, censors, geo-political boundaries, or any feature of the topology which can be expressed at an AS-level. Given the increase in BGP poisoning usage as a security primitive, we set out to evaluate poisoning feasibility in practice beyond simulation. To that end, using an Internet-scale measurement infrastructure, we capture and analyze over 1,400 instances of BGP poisoning across thousands of ASes as a mechanism to maneuver return paths of traffic. We analyze in detail the performance of steering paths, the graph-theoretic aspects of available paths, and re-evaluate simulated systems with this data. We find that the real-world evidence does not completely support the findings from simulated systems published in the literature. We also analyze filtering of BGP poisoning across types of ASes and ISP working groups. We explore the connectivity concerns when poisoning by reproducing a decade old experiment to uncover the current state of an Internet triple the size. We build predictive models for understanding an ASes' vulnerability to poisoning. Finally, an exhaustive measurement of an upper bound on the maximum path length of the Internet is presented, detailing how security research should react to ASes leveraging poisoned long paths. In total, our results and analysis expose the real-world impact of BGP poisoning on past and future security research.

Intra-process memory isolation can improve security by enforcing least-privilege at a finer granularity than traditional operating system controls without the context-switch over-head associated with inter-process communication. A single process can be divided into separate components such that memory belonging to one component can only be accessed by the code of that component. Because the process has traditionally been a fundamental security boundary, assigning different levels of trust to components within a process is a fundamental change in secure systems design. However, so far there has been little research on the challenges of securely implementing intra-process isolation on top of existing operating system abstractions. We identify that despite providing strong intra-process memory isolation, existing, general purpose approaches neglect the ways in which the OS makes memory and other intra-process resources accessible through system objects. Using two recently-proposed memory isolation systems, we show that such designs are vulnerable to generic attacks that bypass memory isolation These attacks use the kernel as a confused deputy, taking advantage of existing intended kernel functionality that is agnostic of intra-process isolation. We argue that the root cause stems from a fundamentally different security model between kernel abstractions and user-level, intra-process memory isolation. Finally, we discuss potential mitigations and show that the performance cost of extending a pt race-based sandbox to prevent the new attacks is high, highlighting the need for more efficient system call interception.

In industrial control systems (ICS), attackers inject malicious control-logic into programmable logic controllers (PLCs) to sabotage physical processes, such as nuclear plants, traffic-light signals, elevators, and conveyor belts. For instance, Stuxnet operates by transfering control logic to Siemens S7-300 PLCs over the network to manipulate the motor speed of centrifuges. These devestating attacks are referred to as control-logic injection attacks. Their network traffic, if captured, contains malicious control logic that can be leveraged as a forensic artifact. In this paper, we present Reditus to recover control logic from a suspicious ICS network traffic. Reditus is based on the observation that an engineering software has a built-in decompiler that can transform the control logic into its source-code. Reditus integrates the decompiler with a (previously-captured) set of network traffic from a control-logic to recover the source code of the binary control-logic automatically. We evaluate Reditus on the network traffic of 40 control logic programs transferred from the SoMachine Basic engineering software to a Modicon M221 PLC. Our evaluation successfully demonstrates that Reditus can recover the source-code of a control logic from its network traffic. (C) 2020 The Author(s). Published by Elsevier Ltd on behalf of DFRWS. All rights reserved.

In this paper, we present a scientific evaluation of four prominent malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously- and never-before-seen files? Is it worth purchasing a network-level malware detector? To identify weaknesses, we tested each tool against 3,536 total files (2,554 or 72% malicious, 982 or 28% benign) of a variety of file types, including hundreds of malicious zero-days, polyglots, and APT-style files, delivered on multiple protocols. We present statistical results on detection time and accuracy, consider complementary analysis (using multiple tools together), and provide two novel applications of the recent cost-benefit evaluation procedure of Iannacone & Bridges. While the ML-based tools are more effective at detecting zero-day files and executables, the signature-based tool may still be an overall better option. Both network-based tools provide substantial (simulated) savings when paired with either host tool, yet both show poor detection rates on protocols other than HTTP or SMTP. Our results show that all four tools have near-perfect precision but alarmingly low recall, especially on file types other than executables and office files – 37 polyglot files, were undetected. Priorities for researchers and takeaways for end users are given.

Programmable logic controllers (PLCs) in industrial control systems (ICS) are vulnerable to remote control logic injection attacks. Attackers target the control logic of a PLC to manipulate the behavior of a physical process such as nuclear plants, power grids, and gas pipelines. Control logic attacks have been studied extensively in the literature, including hiding the transfer of a control logic over the network from both packet header-based signatures, and deep packet inspection. For instance, these attacks transfer a control logic code as data, into small fragments (one-byte per packet), that are further padded with noise data. To detect control logic in ICS network traffic, this paper presents Shade, a novel shadow memory technique that observes the network traffic to maintain a local copy of the current state of a PLC memory. To analyze the memory contents, Shade employs a classification algorithm with 42 unique features categorized into five types at different semantic levels of a control logic code, such as number of rungs, number of consecutive decompiled instructions, and n-grams. We then evaluate Shade against control logic injection attacks on two PLCs, Modicon M221 and MicroLogix 1400 from two ICS vendors, Schneider electric and Allen-Bradley, respectively. The evaluation results show that Shade can detect an attack instance (i.e., identifying at least one attack packet during the transfer of a malicious control logic) accurately without any false alarms.

We examine the extent of needless traffic exposure by the routing infrastructure to nations geographically irrelevant to packet transmission. We quantify what countries are geographically logical to observe on a network path traveling between two nations through the use of convex hulls circumscribing major population centers. We then compare that to the nation states observed in over 2.5 billion measured paths. We examine both the entire geographic topology of the Internet and a subset of the topology that a Tor user would typically interact with. We find that 44 of the Internet and 33 expose traffic to one or more nations. Finally, we consider the scenario where countries exercise both legal and physical control over autonomous systems, gaining access to traffic outside of their geographic borders, but carried by organizations that fall under the AS's registered country's legal jurisdiction. At least 49 irrelevant country when considering both the physical and legal countries that a path traverses.

The security of the Internet's routing infrastructure has underpinned much of the past two decades of distributed systems security research. However, the converse is increasingly true. Routing and path decisions are now important for the security properties of systems built on top of the Internet. In particular, BGP poisoning leverages the de facto routing protocol between Autonomous Systems (ASes) to maneuver the return paths of upstream networks onto previously unusable, new paths. These new paths can be used to avoid congestion, censors, geo-political boundaries, or any feature of the topology which can be expressed at an AS-level. Given the increase in BGP poisoning usage as a security primitive, we set out to evaluate poisoning feasibility in practice beyond simulation. To that end, using an Internet-scale measurement infrastructure, we capture and analyze over 1,400 instances of BGP poisoning across thousands of ASes as a mechanism to maneuver return paths of traffic. We analyze in detail the performance of steering paths, the graph-theoretic aspects of available paths, and re-evaluate simulated systems with this data. We find that the real-world evidence does not completely support the findings from simulated systems published in the literature. We also analyze filtering of BGP poisoning across types of ASes and ISP working groups. We explore the connectivity concerns when poisoning by reproducing a decade old experiment to uncover the current state of an Internet triple the size. We build predictive models for understanding an ASes' vulnerability to poisoning. Finally, an exhaustive measurement of an upper bound on the maximum path length of the Internet is presented, detailing how security research should react to ASes leveraging poisoned long paths. In total, our results and analysis expose the real-world impact of BGP poisoning on past and future security research.

Security aspects of SCADA environments and the systems within are increasingly a center of interest to researchers and security professionals. As the rise of sophisticated and nation-state malware targeting such systems flourishes, traditional digital forensics tools struggle to transfer the same capabilities to systems lacking typical volatile memory primitives, monitoring software, and the compatible operating-system primitives necessary for conducting forensic investigations. Even worse, SCADA systems are typically not designed and implemented with security in mind, nor were they purpose-built to monitor and record system data at the granularity associated with traditional IT systems. Rather, these systems are often built to control field devices and drive industrial processes. More succinctly, SCADA systems were not designed with a primary goal of interacting with the digital world. Consequently, forensics investigators well-versed in the world of digital forensics and incident response face an array of challenges that prevent them from conducting effective forensic investigation in environments with vast amounts of critical infrastructure. In order to bring SCADA systems within the reach of the armies of digital forensics professionals and tooling already available, both researchers and practitioners need a guide to the current state-of-the-art techniques, a road-map to the challenges lying on the path forward, and insight into the future directions R&D must move towards. To that end, this paper presents a survey into the literature on digital forensics applied to SCADA systems. We cover not only the challenges to applying digital forensics to SCADA like most other reviews, but also the range of proposed frameworks, methodologies, and actual implementations in literature.

BGP poisoning, a known side-effect of the Border Gateway Protocol, employs BGP's loop detection mechanism to give an Autonomous System inbound path influence. The viability of a range of recent network measurement, censorship-circumvention, and DDoS defense systems depend on the feasibility of BGP Poisoning. Simultaneously, other proposed systems assume the opposite, that BGP Poisoning is infeasible in practice. Building on a foundation of active measurements, we address the growing division in research and operational ideology concerning BGP poisoning. This work explores what is, and is not, feasible on today's Internet. Our results point to the fact that there is truth in both side's arguments. With poisoning, we can change a remote network's best path to our router in over 77% of cases. These results demonstrate that BGP poisoning is viable in practice, but less than prior studies conducted in simulation only. When successful, we observe that poisoning can steer onto 26 unique paths at maximum and nearly 3 unique paths on average. We also explore the prevalence of filtering that prevents poisoned path propagation. We find over 80% of observed ASes from route collectors will propagate a poisoned path of 250 ASes in length in the same manner as a non-poisoned path of normal length. However, we discover and characterize large amounts of ASes that do filter poisoned paths.

In this paper, we present Nyx, the first system to both effectively mitigate modern Distributed Denial of Service (DDoS) attacks regardless of the amount of traffic under adversarial control and function without outside cooperation or an Internet redesign. Nyx approaches the problem of DDoS mitigation as a routing problem rather than a filtering problem. This conceptual shift allows Nyx to avoid many of the common shortcomings of existing academic and commercial DDoS mitigation systems. By leveraging how Autonomous Systems (ASes) handle route advertisement in the existing Border Gateway Protocol (BGP), Nyx allows the deploying AS to achieve isolation of traffic from a critical upstream AS off of attacked links and onto alternative, uncongested, paths. This isolation removes the need for filtering or de-prioritizing attack traffic. Nyx controls outbound paths through normal BGP path selection, while return paths from critical ASes are controlled through the use of specific techniques we developed using existing traffic engineering principles and require no outside coordination. Using our own realistic Internet-scale simulator, we find that in more than 98% of cases our system can successfully route critical traffic around network segments under transit-link DDoS attacks; a new form of DDoS attack where the attack traffic never reaches the victim AS, thus invaliding defensive filtering, throttling, or prioritization strategies. More significantly, in over 95% of those cases, the alternate path provides complete congestion relief from transitlink DDoS. Nyx additionally provides complete congestion relief in over 75% of cases when the deployer is being directly attacked.

In this work, we take a fundamentally different approach to the problem of analyzing a device for compromises via malware; our approach is OS and instruction architecture independent and relies only on having the raw binary data extracted from the memory dump of a device. Our system leverages a multi-hundred TB dataset of both compromised host memory dumps extracted from the MalRec dataset [8] and the first known dataset of benign host memory dumps running normal, non-compromised software. After an average of 30 to 45 seconds of pre-processing on a single memory dump, our system leverages both traditional machine learning and deep learning algorithms to achieve an average of 98% accuracy of detecting a compromised host.

In this paper we present three attacks against the predictable solo mining (PSM) scheme. In PSM, miners receive shares for submitting partially valid solutions to the current Proof of Work, adding those shares to their account. When the pool successfully mines a block, the block is awarded to the miner with the most shares, and the rewarded miner “pays” an amount of shares equal to the next highest miner’s to claim the block. Our attacks take advantage of the fact that the amount of shares expended winning two different blocks, which have the same monetary value, can vary by up to a factor of four. We show that by strategically spreading its shares across multiple accounts, a malicious miner can generate more revenue than a naive miner of the same computational power by only claiming blocks with a low share cost. By doing so, a miner can reduce computational power it must expend to win a block by more than 30%. Our other two attacks reduce the profitability of victim miners in the pool by minimizing the gap between first and second place when the victim wins a block. This drives up the average amount of computational power the victim must contribute to receive a reward. An adversary not concerned with cost can reduce the number of shares a victim retains after winning a block by up to 26%. We also find that an adversary with more computational power than their victim can reduce the number of shares the victim retains after winning a block by more than 8% with only limited impact on the adversary’s profitability.