Cloud providers' support for network evasion techniques that misrepresent the server's domain name is more prevalent than previously believed, which has serious implications for security and privacy due to the reliance on domain names in common security architectures. Domain fronting is one such evasive technique used by privacy enhancing technologies and malware to hide the domains they visit, and it uses shared hosting and HTTPS to present a benign domain to observers while signaling the target domain in the encrypted HTTP request. In this paper, we construct an ontology of domain name misinformation and detail a novel measurement methodology to identify support among cloud infrastructure providers. Despite several of the largest cloud providers having publicly stated that they no longer support domain fronting, our findings demonstrate a more complex environment with many exceptions. We also present a novel and straightforward attack that allows an adversary to man-in-the-middle all the victim's encrypted traffic bound to a content delivery network that supports domain fronting, breaking the authenticity, confidentiality, and integrity guarantees expected by the victim when using HTTPS. By using dynamic linker hijacking to rewrite the HTTP Host field, our attack does not generate any artifacts that are visible to the victim or passive network monitoring solutions, and the attacker does not need a separate channel to exfiltrate data or perform command-and-control, which can be achieved by rewriting HTTP headers.

Defending an enterprise network requires making prioritization decisions daily; one is deciding which compromised hosts to remediate (reimage). We study the utility of endpoint monitoring data to perform this prioritization, with the driving goal being to minimize “regret” as measured by future (next-week) malware execution on hosts whose remediation was deprioritized. Leveraging data gathered by the vendor of a major endpoint protection product, we show that it is possible to prioritize remediation by training a classifier that predicts imminent malware execution. Perhaps surprisingly, while it might seem essential to maximize the amount of training data by collecting across an array of enterprises to which endpoint protection is deployed, at least in the case of the endpoint protection vendor (itself a major, worldwide company), predictive performance for a single enterprise can remain excellent when training is restricted to the enterprise itself. One advantage of single-enterprise training is the ease of combining different views of the hosts, such as via file-based and network-based monitoring. In the cases studied, although an exact comparison was impossible due to a time gap, the single-enterprise dataset with richer features resulted in superior prediction of malware execution compared to the multi-enterprise dataset.

Network fingerprinting is used to identify applications, provide insight into network traffic, and detect malicious activity. With the broad adoption of TLS, traditional fingerprinting techniques that rely on clear-text data are no longer viable. TLS-specific techniques have been introduced that create a fingerprint string from carefully selected data features in the client_hello to facilitate process identification before data is exchanged. Unfortunately, this approach fails in practice because hundreds of processes can map to the same fingerprint string. We solve this problem by presenting a TLS fingerprinting system that makes use of the destination address, port, and server name in addition to a carefully constructed fingerprint string. The destination context is used to disambiguate the set of processes that match a fingerprint string by applying a weighted naive Bayes classifier, resulting in far greater performance.

The Transport Layer Security (TLS) protocol has evolved in response to different attacks and is increasingly relied on to secure Internet communications. Web browsers have led the adoption of newer and more secure cryptographic algorithms and protocol versions, and thus improved the security of the TLS ecosystem. Other application categories, however, are increasingly using TLS, but too often are relying on obsolete and insecure protocol options. To understand in detail what applications are using TLS, and how they are using it, we developed a novel system for obtaining process information from end hosts and fusing it with network data to produce a TLS fingerprint knowledge base. This data has a rich set of context for each fingerprint, is representative of enterprise TLS deployments, and is automatically updated from ongoing data collection. Our dataset is based on 471 million endpoint-labeled and 8 billion unlabeled TLS sessions obtained from enterprise edge networks in five countries, plus millions of sessions from a malware analysis sandbox. We actively maintain an open source dataset that, at 4,500+ fingerprints and counting, is both the largest and most informative ever published. In this paper, we use the knowledge base to identify trends in enterprise TLS applications beyond the browser: application categories such as storage, communication, system, and email. We identify a rise in the use of TLS by nonbrowser applications and a corresponding decline in the fraction of sessions using version 1.3. Finally, we highlight the shortcomings of naïvely applying TLS fingerprinting to detect malware, and we present recent trends in malware's use of TLS such as the adoption of cipher suite randomization.

We present new analytic techniques for inferring HTTP semantics from passive observations of HTTPS that can infer the value of important fields including the status-code, Content-Type, and Server, and the presence or absence of several additional HTTP header fields, e.g., Cookie and Referer. Our goals are to improve the understanding of the confidentiality limitations of HTTPS, and to explore benign uses of traffic analysis that could replace HTTPS interception and static private keys in some scenarios. We found that our techniques increase the efficacy of malware detection, but they do not enable more powerful website fingerprinting attacks against Tor. Our broader set of results raises concerns about the confidentiality goals of TLS relative to a user's expectation of privacy, warranting future research. We apply our methods to the semantics of both HTTP/1.1 and HTTP/2 on data collected from automated runs of Firefox 58.0, Chrome 63.0, and Tor Browser 7.0.11 in a lab setting, and from applications running in a malware sandbox. We obtain ground truth plaintext for a diverse set of applications from the malware sandbox by extracting the key material needed for decryption from RAM post-execution. We developed an iterative approach to simultaneously solve several multi-class (field values) and binary (field presence) classification problems, and we show that our inference algorithm achieves an unweighted $F_1$ score greater than 0.900 for most HTTP fields examined.

Previous work has intensely studied the prevention and detection of malicious network traffic, but current solutions still lack the efficacy needed to detect Remote Access Trojan (RAT) network activity. This deficiency is becoming more of a threat with the releases of open-source implementations that emphasize ease of use while maintaining stealth and modularity. In this paper, we provide a detailed design and analysis of network-based methods that can detect generic RAT behaviors such as polling, and specific detection techniques targeting three popular open-source RATs: Metasploit, Empire, and Pupy. Our methods rely on passive monitoring as well as semi-active scans targeting suspicious servers that are triggered by the passive monitoring system. Our complete classification system achieves a ~98.5% true positive rate and a ~0.01% false positive rate, validating our approach to RAT detection.

The use of TLS by malware poses new challenges to network threat detection because traditional pattern-matching techniques can no longer be applied to its messages. However, TLS also introduces a complex set of observable data features that allow many inferences to be made about both the client and the server. We show that these features can be used to detect and understand malware communication, while at the same time preserving the privacy of the benign uses of encryption. These data features also allow for accurate malware family attribution of network communication, even when restricted to a single, encrypted flow. To demonstrate this, we performed a detailed study of how TLS is used by malware and enterprise applications. We provide a general analysis on millions of TLS encrypted flows, and a targeted study on 18 malware families composed of thousands of unique malware samples and tens-of-thousands of malicious TLS flows. Importantly, we identify and accommodate for the bias introduced by the use of a malware sandbox. We show that the performance of a malware classifier is correlated with a malware family’s use of TLS, i.e., malware families that actively evolve their use of cryptography are more difficult to classify. We conclude that malware’s usage of TLS is distinct in an enterprise setting, and that these differences can be effectively used in rules and machine learning classifiers.

Passive operating system fingerprinting reveals valuable information to the defenders of heterogeneous private networks; at the same time, attackers can use fingerprinting to reconnoiter networks, so defenders need obfuscation techniques to foil them. We present an effective approach for passive fingerprinting that uses data features from TLS as well as the TCP/IP and HTTP protocols in a multi-session model, which is applicable whenever several sessions can be observed within a time window. In experiments on a real-world private network, our approach identified operating system major and minor versions with accuracies of 99.4% and 97.5%, respectively, and provided significant information gain. We also show that obfuscation strategies can often be defeated due to the difficulty of manipulating data features from all protocols, especially TLS, by studying how obfuscation affects our fingerprinting system. Because devices running unpatched operating systems on private networks create significant vulnerabilities, their detection is critical; our approach achieved over 98% accuracy at this important goal.

The application of machine learning for the detection of malicious network traffic has been well researched over the past several decades; it is particularly appealing when the traffic is encrypted because traditional pattern-matching approaches cannot be used. Unfortunately, the promise of machine learning has been slow to materialize in the network security domain. In this paper, we highlight two primary reasons why this is the case: inaccurate ground truth and a highly non-stationary data distribution. To demonstrate and understand the effect that these pitfalls have on popular machine learning algorithms, we design and carry out experiments that show how six common algorithms perform when confronted with real network data.With our experimental results, we identify the situations in which certain classes of algorithms underperform on the task of encrypted malware traffic classification. We offer concrete recommendations for practitioners given the real-world constraints outlined. From an algorithmic perspective, we find that the random forest ensemble method outperformed competing methods. More importantly, feature engineering was decisive; we found that iterating on the initial feature set, and including features suggested by domain experts, had a much greater impact on the performance of the classification system. For example, linear regression using the more expressive feature set easily outperformed the random forest method using a standard network traffic representation on all criteria considered. Our analysis is based on millions of TLS encrypted sessions collected over 12 months from a commercial malware sandbox and two geographically distinct, large enterprise networks.

Bayesian networks have been used extensively to model and discover dependency relationships among sets of random variables. We learn Bayesian network structure with a combination of human knowledge about the partial ordering of variables and statistical inference of conditional dependencies from observed data. Our approach leverages complementary information from human knowledge and inference from observed data to produce networks that reflect human beliefs about the system as well as to fit the observed data. Applying prior beliefs about partial orderings of variables is an approach distinctly different from existing methods that incorporate prior beliefs about direct dependencies (or edges) in a Bayesian network. We provide an efficient implementation of the partial‐order prior in a Bayesian structure discovery learning algorithm, as well as an edge prior, showing that both priors meet the local modularity requirement necessary for an efficient Bayesian discovery algorithm. In benchmark studies, the partial‐order prior improves the accuracy of Bayesian network structure learning as well as the edge prior, even though order priors are more general. Our primary motivation is in characterizing the evolution of families of malware to aid cyber security analysts. For the problem of malware phylogeny discovery, we find that our algorithm, compared to existing malware phylogeny algorithms, more accurately discovers true dependencies that are missed by other algorithms.

Identifying threats contained within encrypted network traffic poses a unique set of challenges. It is important to monitor this traffic for threats and malware, but do so in a way that maintains the integrity of the encryption. Because pattern matching cannot operate on encrypted data, previous approaches have leveraged observable metadata gathered from the flow, e.g., the flow's packet lengths and inter-arrival times. In this work, we extend the current state-of-the-art by considering a data omnia approach. To this end, we develop supervised machine learning models that take advantage of a unique and diverse set of network flow data features. These data features include TLS handshake meta data, DNS contextual flows linked to the encrypted flow, and the HTTP headers of HTTP contextual flows from the same source IP address within a 5 minute window.We begin by exhibiting the differences between malicious and benign traffic's use of TLS, DNS, and HTTP on millions of unique flows. This study is used to design the feature sets that have the most discriminatory power. We then show that incorporating this contextual information into a supervised learning system significantly increases performance at a 0.00% false discovery rate for the problem of classifying encrypted, malicious flows. We further validate our false positive rate on an independent, real-world dataset.

Traditional flow monitoring provides a high-level view of network communications by reporting the addresses, ports, and byte and packet counts of a flow. This data is valuable, but it gives little insight into the actual content or context of a flow. To obtain this missing insight, we investigated intra-flow data, that is, information about events that occur inside of a flow that can be conveniently collected, stored, and analyzed within a flow monitoring framework. The focus of our work is on new types of data that are independent of protocol details, such as the lengths and arrival times of messages within a flow. These data elements have the attractive property that they apply equally well to both encrypted and unencrypted flows. Protocol-aware telemetry, specifically TLS-aware telemetry, is also analyzed.In this paper, we explore the benefits of enhanced telemetry, desirable properties of new intra-flow data features with respect to a flow monitoring system, and how best to use machine learning classifiers that operate on this data. We provide results on millions of flows processed by our open source program. Finally, we show that leveraging appropriate data features and simple machine learning models can successfully identify threats in encrypted network traffic.

Statistical detection of mass malware has been shown to be highly successful. However, this type of malware is less interesting to cyber security officers of larger organizations, who are more concerned with detecting malware indicative of a targeted attack. Here we investigate the potential of statistically based approaches to detect such malware using a malware family associated with a large number of targeted network intrusions. Our approach is complementary to the bulk of statistical based malware classifiers, which are typically based on measures of overall similarity between executable files. One problem with this approach is that a malicious executable that shares some, but limited, functionality with known malware is likely to be misclassified as benign. Here a new approach to malware classification is introduced that classifies programs based on their similarity with known malware subroutines. It is illustrated that malware and benign programs can share a substantial amount of code, implying that classification should be based on malicious subroutines that occur infrequently, or not at all in benign programs. Various approaches to accomplishing this task are investigated, and a particularly simple approach appears the most effective. This approach simply computes the fraction of subroutines of a program that are similar to malware subroutines whose likes have not been found in a larger benign set. If this fraction exceeds around 1.5 %, the corresponding program can be classified as malicious at a 1 in 1000 false alarm rate. It is further shown that combining a local and overall similarity based approach can lead to considerably better prediction due to the relatively low correlation of their predictions.

While an immense amount of work has gone into novel clustering algorithms, little work has focused on developing compact, domain-specific explanations for the results of the clustering algorithms. Attaching semantic meaning to a cluster has numerous benefits, including the ability for such a description to be both human and machine readable. In this paper, we assume that the clusters are given to us, and find the minimal set of features that can differentiate one cluster from the remaining set of samples. We formulate this problem as an integer linear program. By using samples not belonging to the cluster in the optimization formulation, the resulting description will be minimal and contain no false positives. The efficacy of this method is demonstrated on simulation data and real-world malware data run in a sandbox that collects behavioral characteristics. In the case of malware, once it has been clustered, it would have been sent to a reverse engineer who would have been tasked with creating the actual meaning of the clustering results and disseminating this information through signatures or indicators of compromise. This is a time-consuming process that can take hours to weeks depending on the complexity of the malware family. The methods presented in this paper automatically generate optimal signatures, which can then be quickly propagated to help contain the spread of a malware family.

Malware phylogenetics help cybersecurity experts to quickly understand a new malware sample by placing the new sample in the context of similar samples that have been previously reverse engineered. Recently, researchers have begun using malware code as data to infer directed acyclic graphs (DAG) that model the evolutionary relationships among samples of malware. A DAG is the ideal model for a phylogenetic graph because it includes the merges and branches that are often present in malware evolution. We present a novel Bayesian network discovery algorithm for learning a DAG via statistical inference of conditional dependencies from observed data with an informative prior on the partial ordering of variables. Our approach leverages the information on edge direction that a human can provide and the edge presence inference which data can provide. We give an efficient implementation of the partial-order prior in a Bayesian structure discovery learning algorithm, as well as a related structure prior, showing that both priors meet the local modularity requirement necessary for the efficient Bayesian discovery algorithm. We apply our algorithm to learn phylogenetic graphs on three malicious families and two benign families where the ground truth is known; and show that compared to competing algorithms, our algorithm more accurately identifies directed edges.

Computer security requires statistical methods to quickly and accurately flag malicious programs. This article proposes a nonparametric Bayesian approach for classifying programs as benign or malicious and simultaneously clustering malicious programs. The analysis is based on the dynamic trace (DT) of instructions under the first-order Markov assumption. Each row of the trace's transition matrix is modeled using the Dirichlet process mixture (DPM) model. The DPM model clusters programs within each class (malicious or benign), and produces the posterior probability of being a malware which is used for classification. The novelty of the model is using this clustering algorithm to improve the classification accuracy. The simulation study shows that the DPM model outperforms the elastic net logistic (ENL) regression and the support vector machine (SVM) in classification performance under most of the scenarios, and also outperforms the spectral clustering method for grouping similar malware. In an analysis of real malicious and benign programs, the DPM model gives significantly better classification performance than the ENL model, and competitive results to the SVM. More importantly, the DPM model identifies clusters of programs during the classification procedure which is useful for reverse engineering.

The publication and sharing of network trace data is a critical to the advancement of collaborative research among various entities, both in government, private sector, and academia. However, due to the sensitive and confidential nature of the data involved, entities have to employ various anonymization techniques to meet legal requirements in compliance with confidentiality policies. Nevertheless, the very composition of network trace data makes it a challenge when applying anonymization techniques. On the other hand, basic application of microdata anonymization techniques on network traces is problematic and does not deliver the necessary data usability. Therefore, as a contribution, we point out some of the ongoing challenges in the network trace anonymization. We then suggest usability-aware anonymization heuristics by employing microdata privacy techniques while giving consideration to usability of the anonymized data. Our preliminary results show that with trade-offs, it might be possible to generate anonymized network traces with enhanced usability, on a case-by-case basis using micro-data anonymization techniques.

Malware phylogenetics has gained a lot of traction over the past several years. More recently, researchers have begun looking at directed acyclic graphs (DAG) to model the evolutionary relationships between samples of malware. Phylogenetic graphs offer analysts a better understanding of how malware has evolved by clearly illustrating the lineage of a given family. In this paper, we present a novel algorithm based on graphical lasso. We extend graphical lasso to incorporate multiple views, both static and dynamic, of malware. For each program family, a convex combination of the views is found such that the objective function of graphical lasso is maximized. Learning the weights of each view on a per-family basis, as opposed to treating all views as an extended feature vector, is essential in the malware domain because different families employ different obfuscation strategies which limits the information of different views. We demonstrate results on three malicious families and two benign families where the ground truth is known.

Malicious software (malware) has become a prominent fixture in computing. There have been many methods developed over the years to combat the spread of malware, but these methods have inevitably been met with countermeasures. For instance, signature-based malware detection gave rise to polymorphic viruses. This “arms race” will undoubtedly continue for the foreseeable future as the incentives to develop novel malware continue to outweigh the costs. In this dissertation, I describe analysis frameworks for three important problems related to malware: classification, clustering, and phylogenetic reconstruction. The important component of my methods is that they all take into account multiple views of malware. Typically, analysis has been performed in either the static domain (e.g. the byte information of the executable) or the dynamic domain (e.g. system call traces). This dissertation develops frameworks that can easily incorporate well-studied views from both domains, as well as any new views that may become popular in the future. The only restriction that must be met is that a positive semidefinite similarity (kernel) matrix must be defined on the view, a restriction that is easily met in practice.