While human factors in the privacy of fertility tracking apps – health trackers that record user's menstrual or pregnancy data – has been the subject of extensive study, little attention has been paid to the technical aspects of apps' data handling practices. We conduct a network-based measurement study of a corpus of 20 Android fertility tracking apps from the Google Play Store, focusing on how user data is shared with third party advertising services. After systematizing app features, we conduct a series of standardized user interactions across all apps in an environment that records TLS-stripped network traffic. In a subset of apps (n=5) we identify explicit leakage of user health data as well implicit leakage through highly targeted contextual advertising URL's. Equally importantly, we observe additional apps that use an ad-based monetization model without apparent leakage of user data, as well as several apps the interact only minimally with ad services. These findings provide technical grounding for widespread user concerns, but also underscore the importance of consumer choice in the privacy implications of app-based fertility tracking.

Endpoint threat detection research hinges on the availability of worthwhile evaluation benchmarks, but experimenters' understanding of the contents of benchmark datasets is often limited. Typically, attention is only paid to the realism of attack behaviors, which comprises only a small percentage of the audit logs in the dataset, while other characteristics of the data are inscrutable and unknown. We propose a new set of questions for what to talk about when we talk about logs (i.e., datasets): What activities are in the dataset? We introduce a novel visualization that succinctly represents the totality of 100+ GB datasets by plotting the occurrence of provenance graph neighborhoods in a time series. How synthetic is the background activity? We perform autocorrelation analysis of provenance neighborhoods in the training split to identify process behaviors that occur at predictable intervals in the test split. Finally, How conspicuous is the malicious activity? We quantify the proportion of attack behaviors that are observed as benign neighborhoods in the training split as compared to previously-unseen attack neighborhoods. We then validate these questions by profiling the classification performance of state-of-the-art intrusion detection systems (R-CAID, FLASH, KAIROS, GNN) against a battery of public benchmark datasets (DARPA Transparent Computing and OpTC, ATLAS, ATLASv2). We demonstrate that synthetic background activities dramatically inflate True Negative Rates, while conspicuous malicious activities artificially boost True Positive Rates. Further, by explicitly controlling for these factors, we provide a more holistic picture of classifier performance. This work will elevate the dialogue surrounding threat detection datasets and will increase the rigor of threat detection experiments.

While Endpoint Detection and Response (EDR) are able to efficiently monitor threats by comparing static rules to the event stream, their inability to incorporate past system context leads to high rates of false alarms. Recent work has demonstrated Provenance-based Intrusion Detection Systems (Prov-IDS) that can examine the causal relationships between abnormal behaviors to improve threat classification. However, employing these Prov-IDS in practical settings remains difficult – state-of-the-art neural network based systems are only fast in a fully offline deployment model that increases attacker dwell time, while simultaneously using simplified and less accurate provenance graphs to reduce memory consumption. Thus, today's Prov-IDS cannot operate effectively in the real-time streaming setting required for commercial EDR viability. This work presents the design and implementation of ORCHID, a novel Prov-IDS that performs fine-grained detection of process-level threats over a real time event stream. ORCHID takes advantage of the unique immutable properties of a versioned provenance graphs to iteratively embed the entire graph in a sequential RNN model while only consuming a fraction of the computation and memory costs. We evaluate ORCHID on four public datasets, including DARPA TC, to show that ORCHID can provide competitive classification performance while eliminating detection lag and reducing memory consumption by two orders of magnitude.

Data collection without proper consent is a growing concern as smart home devices gain prevalence. It is especially difficult to obtain consent from incidental users because they may be unaware or feel pressured to consent. To understand what appropriate consent means in smart homes, we conducted an online survey (N=360) covering 6 common consent facets: freely given, revertible, informed, enthusiastic, specific, and unburdensome. We study how these facets affect perceived acceptability of data collection and how users would allocate responsibility for obtaining consent. Our results show that all facets have meaningful impacts on perceived acceptability of data collection, and eroding freely given had the greatest impact. Device owners were considered the most responsible for obtaining consent. Based on these findings, we provide recommendations for users, device manufacturers, and policymakers to improve consent practices in smart homes, such as designing consent interfaces that prioritize multiple facets of consent.

MITRE ATT&CK is an open-source taxonomy of adversary tactics, techniques, and procedures based on real-world observations. Increasingly, organizations leverage ATT&CK technique "coverage" as the basis for evaluating their security posture, while Endpoint Detection and Response (EDR) and Security Indicator and Event Management (SIEM) products integrate ATT & CK into their design as well as marketing. However, the extent to which ATT & CK coverage is suitable to serve as a security metric remains unclear— Does ATT&CK coverage vary meaningfully across different products? Is it possible to achieve total coverage of ATT&CK? Do endpoint products that detect the same attack behaviors even claim to cover the same ATT & CK techniques? In this work, we attempt to answer these questions by conducting a comprehensive (and, to our knowledge, the first) analysis of endpoint detection products' use of MITRE ATT & CK. We begin by evaluating 3 ATT & CK-annotated detection rulesets from major commercial providers (Carbon Black, Splunk, Elastic) and a crowdsourced ruleset (Sigma) to identify commonalities and underutilized regions of the ATT & CK matrix. We continue by performing a qualitative analysis of unimplemented ATT & CK techniques to determine their feasibility as detection rules. Finally, we perform a consistency analysis of ATT & CK labeling by examining 37 specific threat entities for which at least 2 products include specific detection rules. Combined, our findings highlight the limitations of overdepending on ATT & CK coverage when evaluating security posture; most notably, many techniques are unrealizable as detection rules, and coverage of an ATT & CK technique does not consistently imply coverage of the same real-world threats.

ATLASv2 is based on a previously generated dataset included in "ATLAS: A Sequence-based Learning Approach for Attack Investigation." The original ATLAS dataset is comprised of Windows Security Auditing system logs, Firefox logs, and DNS logs via WireShark. In ATLASv2, we aim to enrich the ATLAS dataset with higher quality background noise and additional logging vantage points. This work replicates the ten attack scenarios described in ATLAS, but extends the logging to include Sysmon logs and events tracked through VMware Carbon Black Cloud. The main contribution of ATLASv2 is to improve the quality of the benign system activity and the integration of the attack scenarios. Instead of relying on automated scripts to generate activity, we had two researchers use the victim machines as their primary work stations throughout the course of the engagement. This allowed us to capture system logs on actual user behavior. Additionally, the researchers conducted the attacks in a lab setup allowing the integration of the attack into the work flow of the victim user. This allows the ATLASv2 dataset to provide realistic system logs that mirror the system log activity generated in real-world attacks.

Serverless computing is supplanting past versions of cloud computing as the easiest way to rapidly prototype and deploy applications. However, the reentrant and ephemeral nature of serverless functions only exacerbates the challenge of correctly specifying security policies. Unfortunately, with role-based access control solutions like Amazon Identity and Access Management (IAM) already suffering from pervasive misconfiguration problems, the likelihood of policy failures in serverless applications is high. In this work, we introduce GRASP, a graph-based analysis framework for modeling serverless access control policies as queryable reachability graphs. GRASP generates reusable models that represent the principals of a serverless application and the interactions between those principals. We implement GRASP for Amazon IAM in Prolog, then deploy it on a corpus of 731 open source Amazon Lambda applications. We find that serverless policies tend to be short and highly permissive, e.g., 92% of surveyed policies are comprised of just 10 statements and 30% exhibit full reachability between all application functions and resources. We then use GRASP to identify potential attack vectors permitted by these policies, including hundreds of sensitive access channels, a dozen publicly-exposed resources, and four channels that may permit an attacker to exfiltrate an application's private resources through one of its public resources. These findings demonstrate GRASP's utility as a means of identifying opportunities for hardening application policies and highlighting potential exfiltration channels.

The increasing complexity of attacks has given rise to varied security applications tackling profound tasks, ranging from alert triage to attack reconstruction. Yet, security products, such as Endpoint Detection and Response, bring together applications that are developed in isolation, trigger many false positives, miss actual attacks, and produce limited labels useful in supervised learning schemes. To address these challenges, we propose DrSec-a system employing self-supervised learning to pre-train foundation language models (LMs) that ingest event-sequence data and emit distributed representations for processes. Once pre-trained, the LMs can be adapted to solve different downstream tasks with limited to no supervision, helping unify the currently fractured application ecosystem. We trained DrSec with two LM types on a real-world dataset containing similar to 91M processes and similar to 2.55B events, and tested it in three application domains. We found that DrSec enables accurate, unsupervised process identification; outperforms leading methods on alert triage to reduce alert fatigue (e.g., 75.11% vs. similar to 64.31% precision-recall area under curve); and accurately learns expert-developed rules, allowing tuning incident detectors to control false positives and negatives.

"Alert fatigue" is one of the biggest challenges faced by the Security Operations Center (SOC) today, with analysts spending more than half of their time reviewing false alerts. Endpoint detection products raise alerts by pattern matching on event telemetry against behavioral rules that describe potentially malicious behavior, but can suffer from high false positives that distract from actual attacks. While alert triage techniques based on data provenance may show promise, these techniques can take over a minute to inspect a single alert, while EDR customers may face tens of millions of alerts per day; the current reality is that these approaches aren't nearly scalable enough for production environments. We present Carbon Filter, a statistical learning based system that dramatically reduces the number of alerts analysts need to manually review. Our approach is based on the observation that false alert triggers can be efficiently identified and separated from suspicious behaviors by examining the process initiation context (e.g., the command line) that launched the responsible process. Through the use of fast-search algorithms for training and inference, our approach scales to millions of alerts per day. Through batching queries to the model, we observe a theoretical maximum throughput of 20 million alerts per hour. Based on the analysis of tens of million alerts from customer deployments, our solution resulted in a 6-fold improvement in the Signal-to-Noise ratio without compromising on alert triage performance.

In modern enterprise security, endpoint detection products fire an alert when process activity matches known attack behavior patterns. Human analysts then perform Root Cause Analysis (RCA) over event logs to determine if the alert is indicative of an actual attack. Data Provenance can help to automate RCA by representing event logs as a causal dependency graphs; in fact, researchers are now examining whether provenance-based anomaly detection should replace pattern-based detection altogether. Unfortunately, we observe that current approaches leverage off-the-shelf graph embedding techniques that are unable to associate events with their root causes. This shortcoming not only fails to capitalize on the RCA capabilities of provenance, but also leaves provenance-based IDS vulnerable to mimicry and evasion attacks. This work presents the design and implementation of R-CAID, a novel approach to incorporate RCA into provenancebased IDS. R- CAID precomputes each node's root causes during graph construction, then directly links those nodes to their root causes during embedding. Further, R-CAID's classification model is node/process-level, rather than graph/system-level, bringing it more in line with the precision of commercial systems. Under a passive adversary model, we find that R-CAID consistently outperforms baseline graph neural networks, sequence-based log IDS, and even a commercial endpoint detection system. Under a white-box active adversary model, R-CAID maintains a high level of performance (e.g., for DARPA Theia, 0.94 AUC adversarial down from 0.99 passive). R-CAID achieves this by associating each system entity with its immutable and unforgeable root causes, preventing adversaries from being able to masquerade as legitimate processes. This work is thus the first to demonstrate the promise of provenance-based IDS in a manner that avoids the pitfalls of mimicry and evasion.

—Despite their promising experimental performance, intrusion detection systems in practice are beset by false positives, leading to threat alert fatigue. We believe a fundamental cause of this disconnect is that datasets used to evaluate anomaly detection systems fail to meet the assumption that the benign data is representative of normal user behaviors. To analyze this issue, we present a method to visualize the behaviors system log datasets and apply it to various commonly-used datasets, as well as a VM used as one of our workstations to generate an example log containing realistic user behaviors. We then present a method to uncover the periodicity of behaviors arising from workload generators used in certain datasets.

Exploration of Internet of Things (IoT) security often focuses on threats posed by external and technically-skilled attackers. While it is important to understand these most extreme cases, it is equally important to understand the most likely risks of harm posed by smart device ownership. In this paper, we explore how smart devices are misused - used without permission in a manner that causes harm - by device owners' everyday associates such as friends, family, and romantic partners. In a preliminary characterization survey (n = 100), we broadly capture the kinds of unauthorized use and misuse incidents participants have experienced or engaged in. Then, in a prevalence survey (n = 483), we assess the prevalence of these incidents in a demographicallyrepresentative population. Our findings show that unauthorized use of smart devices is widespread (experienced by 43% of participants), and that misuse is also common (experienced by at least 19% of participants). However, highly individual factors determine whether these unauthorized use events constitute misuse. Through a focus on everyday abuses, this work sheds light on the most prevalent security and privacy threats faced by smart-home owners today.

Auditing, a central pillar of operating system security, has only recently come into its own as an active area of public research. This resurgent interest is due in large part to the notion of data provenance, a technique that iteratively parses audit log entries into a dependency graph that explains the history of system execution. Provenance facilitates precise threat detection and investigation through causal analysis of sophisticated intrusion behaviors. However, the absence of a foundational audit literature, combined with the rapid publication of recent findings, makes it difficult to gain a holistic picture of advancements and open challenges in the area.In this work, we survey and categorize the provenance-based system auditing literature, distilling contributions into a layered taxonomy based on the audit log capture and analysis pipeline. Recognizing that the Reduction Layer remains a key obstacle to the further proliferation of causal analysis technologies, we delve further on this issue by conducting an ambitious independent evaluation of 8 exemplar reduction techniques against the recently-released DARPA Transparent Computing datasets. Our experiments uncover that past approaches frequently prune an overlapping set of activities from audit logs, reducing the synergistic benefits from applying them in tandem; further, we observe an inverse relation between storage efficiency and anomaly detection performance. However, we also observe that log reduction techniques are able to synergize effectively with data compression, potentially reducing log retention costs by multiple orders of magnitude. We conclude by discussing promising future directions for the field.

Reliable methods for host-layer intrusion detection remained an open problem within computer security. Recent research has recast intrusion detection as a provenance graph anomaly detection problem thanks to concurrent advancements in machine learning and causal graph auditing. While these approaches show promise, their robustness against an adaptive adversary has yet to be proven. In particular, it is unclear if mimicry attacks, which plagued past approaches to host intrusion detection, have a similar effect on modern graph-based methods. In this work, we reveal that systematic design choices have allowed mimicry attacks to continue to abound in provenance graph host intrusion detection systems (Prov-HIDS). Against a corpus of exemplar Prov-HIDS, we develop evasion tactics that allow attackers to hide within benign process behaviors. Evaluating against public datasets, we demonstrate that an attacker can consistently evade detection (100% success rate) without modifying the underlying attack behaviors. We go on to show that our approach is feasible in live attack scenarios and outperforms domain-general adversarial sample techniques. Through open sourcing our code and datasets, this work will serve as a benchmark for the evaluation of future Prov-HIDS.

—MITRE ATT&CK is an open source taxonomy of adversary tactics, techniques, and procedures based on real-world observations. Increasingly, organizations leverage the ATT&CK as the basis for evaluating their security posture, while Endpoint Detection & Response (EDR) products have integrated ATT&CK into their design and marketing. However, the extent to which this integration has improved real-world security remains unclear – Does increasing your organization’s coverage of ATT&CK improve its security? In this work, we attempt to answer this question by conducting a comparative analysis of EDR products’ use of the MITRE ATT&CK knowledge base. We begin by evaluating 3 ATT&CK-annotated EDR detection rule sets from major commercial providers (Carbon Black, Splunk, Elastic) to identify commonalities and underutilized regions of the ATT&CK matrix. We continue by performing a complete qualitative analysis of ATT&CK techniques to determine their feasibility as detection rules. Our initial findings indicate potential limitations of using ATT&CK coverage as an evaluation metric for EDR tools, as we identify several techniques that do not have viable endpoint detection strategies.

System auditing is an essential tool for detecting malicious events and conducting forensic analysis. Although used extensively on general-purpose systems, auditing frameworks have not been designed with consideration for the unique constraints and properties of Real-Time Systems (RTS). System auditing could provide tremendous benefits for security-critical RTS. However, a naive deployment of auditing on RTS could violate the temporal requirements of the system while also rendering auditing incomplete and ineffectual. To ensure effective auditing that meets the computational needs of recording complete audit information while adhering to the temporal requirements of the RTS, it is essential to carefully integrate auditing into the real-time (RT) schedule. This work adapts the Linux Audit framework for use in RT Linux by leveraging the common properties of such systems, such as special purpose and predictability. Ellipsis , an efficient system for auditing RTS, is devised that learns the expected benign behaviors of the system and generates succinct descriptions of the expected activity. Evaluations using varied RT applications show that Ellipsis reduces the volume of audit records generated during benign activity by up to 97.55% while recording detailed logs for suspicious activities. Empirical analyses establish that the auditing infrastructure adheres to the properties of predictability and isolation that are important to RTS. Furthermore, the schedulability of RT tasksets under audit is comprehensively analyzed to enable the safe integration of auditing in RT task schedules.

Causality analysis is an effective technique for investigating and detecting cyber attacks. However, by focusing on auditing at the Operating System level, existing causal analysis techniques lack visibility into important application-level semantics, such as configuration changes that control application runtime behavior. This leads to incorrect attack attribution and half-baked tracebacks. In this work, we propose Dossier, a specialized provenance tracker that enhances the visibility of the Linux auditing infrastructure. By providing additional hooks into the system, Dossier can generate a holistic view of the target application’s event history and causal chains, particularly those pertaining to configuration changes that are among the most common attack vectors observed in the real world. The extra vantage points in Dossier enable forensic investigators to bridge the semantic gap and correctly piece together attack fragments. Dossier leverages the versatility of information flow tracking and system call introspection to track all configuration changes, including both dynamic modifications that directly update configuration-related program variables and revisions to configuration files on disk with negligible runtime overhead (less than 7%). Evaluation on realistic workloads and real-world attack scenarios shows that Dossier can effectively reason about configuration-based attacks and accurately reconstruct the whole attack stories.

System logs are invaluable to forensic audits, but grow so large that in practice fine-grained logs are quickly discarded – if captured at all – preventing the real-world use of the provenance-based investigation techniques that have gained popularity in the literature. Encouragingly, forensically-informed methods for reducing the size of system logs are a subject of frequent study. Unfortunately, many of these techniques are designed for offline reduction in a central server, meaning that the up-front cost of log capture, storage, and transmission must still be paid at the endpoints. Moreover, to date these techniques exist as isolated (and, often, closed-source) implementations; there does not exist a comprehensive framework through which the combined benefits of multiple log reduction techniques can be enjoyed. In this work, we present FAuST, an audit daemon for performing streaming audit log reduction at system endpoints. After registering with a log source (e.g., via Linux Audit’s audisp utility), FAuST incrementally builds an in-memory provenance graph of recent system activity. During graph construction, log reduction techniques that can be applied to local subgraphs are invoked immediately using event callback handlers, while techniques meant for application on the global graph are invoked in periodic epochs. We evaluate FAuST, loaded with eight different log reduction modules from the literature, against the DARPA Transparent Computing datasets. Our experiments demonstrate the efficient performance of FAuST and identify certain subsets of reduction techniques that are synergistic with one another. Thus, FAuST dramatically simplifies the evaluation and deployment of log reduction techniques.

System auditing is a powerful tool that provides insight into the nature of suspicious events in computing systems, allowing machine operators to detect and subsequently investigate security incidents. While auditing has proven invaluable to the security of traditional computers, existing audit frameworks are rarely designed with consideration for Real-Time Systems (RTS). The transparency provided by system auditing would be of tremendous benefit in a variety of security-critical RTS domains, (e.g., autonomous vehicles); however, if audit mechanisms are not carefully integrated into RTS, auditing can be rendered ineffectual and violate the real-world temporal requirements of the RTS. In this paper, we demonstrate how to adapt commodity audit frameworks to RTS. Using Linux Audit as a case study, we first demonstrate that the volume of audit events generated by commodity frameworks is unsustainable within the temporal and resource constraints of real-time (RT) applications. To address this, we present Ellipsis, a set of kernel-based reduction techniques that leverage the periodic repetitive nature of RT applications to aggressively reduce the costs of system-level auditing. Ellipsis generates succinct descriptions of RT applications' expected activity while retaining a detailed record of unexpected activities, enabling analysis of suspicious activity while meeting temporal constraints. Our evaluation of Ellipsis, using ArduPilot (an open-source autopilot application suite) demonstrates up to 93% reduction in audit log generation.