Recent years have seen a proliferation of research on adversarial machine learning. Numerous papers demonstrate powerful algorithmic attacks against a wide variety of machine learning (ML) models, and numerous other papers propose defenses that can withstand most attacks. However, abundant real-world evidence suggests that actual attackers use simple tactics to subvert ML-driven systems, and as a result security practitioners have not prioritized adversarial ML defenses. Motivated by the apparent gap between researchers and practitioners, this position paper aims to bridge the two domains. We first present three real-world case studies from which we can glean practical insights unknown or neglected in research. Next we analyze all adversarial ML papers recently published in top security conferences, highlighting positive trends and blind spots. Finally, we state positions on precise and cost-driven threat modeling, collaboration between industry and academia, and reproducible research. We believe that our positions, if adopted, will increase the real-world impact of future endeavours in adversarial ML, bringing both researchers and practitioners closer to their shared goal of improving the security of ML systems.

Online social networks (OSNs) attract attackers that use abusive accounts to conduct malicious activities for economic, political, and personal gain. In response, OSNs often deploy abusive account classifiers using machine learning (ML) approaches. However, a practical, effective ML-based defense requires carefully engineering features that are robust to adversarial manipulation, obtaining enough ground truth labeled data for model training, and designing a system that can scale to all active accounts on an OSN (potentially in the billions). To address these challenges we present Deep Entity Classification (DEC), an ML framework that detects abusive accounts in OSNs that have evaded other, traditional abuse detection systems. We leverage the insight that while accounts in isolation may be difficult to classify, their embeddings in the social graph-the network structure, properties, and behaviors of themselves and those around them-are fundamentally difficult for attackers to replicate or manipulate at scale. Our system: Extracts "deep features" of accounts by aggregating properties and behavioral features from their direct and indirect neighbors in the social graph. Employs a "multi-stage multi-task learning" (MS-MTL) paradigm that leverages imprecise ground truth data by consuming, in separate stages, both a small number of high-precision human-labeled samples and a large amount of lower-precision automated labels. This architecture results in a single model that provides high-precision classification for multiple types of abusive accounts. Scales to billions of users through various sampling and reclassification strategies that reduce system load. DEC has been deployed at Facebook, where it classifies all users continuously, resulting in an estimated reduction of abusive accounts on the network by 27% beyond those already detected by other, traditional methods.

IP address classification and clustering are important tools for security practitioners in understanding attacks and employing proactive defenses. Over the past decade, network providers have begun transitioning from IPv4 to the more flexible IPv6, and a third of users now access online services over IPv6. However, there is no reason to believe that the properties of IPv4 addresses used for security applications should carry over to IPv6, and to date there has not yet been a large-scale study comparing the two protocols at a user (as opposed to a client or address) level. In this paper, we establish empirical grounding on how both ordinary users and attackers use IPv6 in practice, compared with IPv4. Using data on benign and abusive accounts at Facebook, one of the largest online platforms, we conduct user-centric analyses that assess the spatial and temporal properties of users' IP addresses, and IP-centric evaluations that characterize the user populations on IP addresses. We find that compared with IPv4, IPv6 addresses are less populated with users and shorter lived for each user. While both protocols exhibit outlying behavior, we determine that IPv6 outliers are significantly less prevalent and diverse, and more readily predicted. We also study the effects of subnetting IPv6 addresses at different prefix lengths, and find that while /56 subnets are closest in behavior to IPv4 addresses for malicious users, either the full IPv6 address or /64 subnets are most suitable for IP-based security applications, with both providing better performance tradeoffs than IPv4 addresses. Ultimately, our findings provide guidance on how security practitioners can handle IPv6 for applications such as blocklisting, rate limiting, and training machine learning models.

Online social networks (OSNs) such as Facebook, Twitter, and LinkedIn give hundreds of millions of individuals around the world the ability to communicate and build communities. However, the extensive user base of OSNs provides considerable opportunity for malicious actors to abuse the system, with fake accounts generating the vast majority of harmful actions and content. Social networks employ sophisticated detection mechanisms based on machine-learning classifiers and graph analysis to identify and remediate the actions of fake accounts. Disabling or deleting these detected accounts is not tractable when the number of false positives (i.e., real users disabled) is significant in absolute terms. Using challenge-based verification systems such as CAPTCHAs or phone confirmation as a response for detected fake accounts can enable erroneously detected real users to recover their access, while also making it difficult for attackers to abuse the platform. In order to maintain a verification system’s effectiveness over time, it is important to iterate on the system to improve the real user experience and adapt the platform’s response to adversarial actions. However, at present there is no established method to evaluate how effective each iteration is at stopping fake accounts and letting real users through. This paper proposes a method of assessing the effectiveness of experimental iterations for OSN verification systems, and presents an evaluation of this method against humanlabelled ground truth data using production Facebook data. Our method reduces the volume of necessary human labelled data by 70%, decreases the time necessary for classification by 81%, has suitable precision/recall for making decisions in response to experiments, and enables continuous monitoring of the effectiveness of the applied experimental changes.

Malicious actors on social networks often create fake accounts that impersonate legitimate users in order to carry out scams, send spam and phishing messages, or engage in other abusive behavior. The act of assuming a real person»s identity gives the bad actor an additional level of trust that can make the attack more likely to succeed. In addition to the direct damage caused by the fraudulent activity, impersonation erodes users» trust in the platform and damages the brand of the legitimate user being impersonated. In this paper, we present a large-scale system that identifies and takes action on impersonation profiles on a social network. The system is composed of three tiers of protection: (1) an offline system that protects all users by searching for profile data duplicity combined with suspicious signals; (2) an offline system with stricter logic that proactively protects high-profile users in population segments that have historically been targeted for impersonation; and (3) an online system that provides low-latency protection to a manually curated list of individuals that have been actively targeted by impersonation. The tiers build on one another with decreasing detection latency and more targeted user coverage. We have deployed our system on the professional social network LinkedIn and drastically reduced the extent of impersonation abuse on LinkedIn. Specifically, we observed a reduction in the impersonator median time-to-detect of more than $98%$, a more than fourfold improvement in recall (as measured on a monthly basis) and reduction over $95%$ in the number of impersonation reports by the most targeted company on LinkedIn.

BACKGROUND Artificial Intelligence (AI), and Machine Learning (ML) in particular, provide a set of useful analytic and decision-making techniques that are being leveraged by an ever-growing community of practitioners, including applications with security-sensitive elements. However, while security researchers often utilize such techniques to address problems and AI/ML researchers develop techniques for big-data analytics applications, neither community devotes much attention to the other. Within security research, AI/ML components are often regarded as black-box solvers. Conversely, the learning community seldom considers the security/privacy implications entailed in the application of their algorithms when designing them. While these two communities generally focus on different issues, where these two fields do meet, interesting problems appear. Researchers working in the intersection have already raised many novel questions for both communities and created a new branch of research known as secure learning. The past few years have particularly seen increasing interest within the AISec / Secure Learning community. There are several reasons for this surge. Firstly, machine learning, data mining, and other artificial intelligence technologies play a key role in extracting knowledge, situational awareness, and security intelligence from Big Data. Secondly, companies like Google, Amazon and Splunk are increasingly exploring and deploying learning technologies to address Big Data problems for their customers. Finally, these trends are increasingly exposing companies and their customers/users to

Artificial Intelligence (AI) and Machine Learning (ML) provide a set of useful analytic and decision-making techniques that are being leveraged by an ever-growing community of practitioners, including many whose applications have security-sensitive elements. However, while security researchers often utilize such techniques to address problems and AI/ML researchers develop techniques for Big Data analytics applications, neither community devotes much attention to the other. Within security research, AI/ML components are usually regarded as black-box solvers. Conversely, the learning community seldom considers the security/privacy implications entailed in the application of their algorithms when they are designing them. While these two communities generally focus on different directions, where these two fields do meet, interesting problems appear. Researchers working in this intersection have raised many novel questions for both communities and created a new branch of research known as secure learning. The AISec workshop has become the primary venue for this unique fusion of research.

Online social networks (OSNs) are appealing platforms for spammers and fraudsters, who typically use fake or compromised accounts to connect with and defraud real users. To combat such abuse, OSNs allow users to report fraudulent profiles or activity. The OSN can then use reporting data to review and/or limit activity of reported accounts.Previous authors have suggested that an OSN can augment its takedown algorithms by identifying a "trusted set" of users whose reports are weighted more heavily in the disposition of flagged accounts. Such identification would allow the OSN to improve both speed and accuracy of fake account detection and thus reduce the impact of spam on users.In this work we provide the first public, data-driven assessment of whether the above assumption is true: are some users better at reporting than others? Specifically, is reporting skill both measurable, i.e., possible to distinguish from random guessing; and repeatable, i.e., persistent over repeated sampling?Our main contributions are to develop a statistical framework that describes these properties and to apply this framework to data from LinkedIn, the professional social network. Our data includes member reports of fake profiles as well as the more voluminous, albeit weaker, signal of member responses to connection requests. We find that members demonstrating measurable, repeatable skill in identifying fake profiles do exist but are rare: at most 2.4% of those reporting fakes and at most 1.3% of those rejecting connection requests. We conclude that any reliable "trusted set" of members will be too small to have noticeable impact on spam metrics.

Artificial Intelligence (AI) and Machine Learning (ML) provide a set of useful analytic and decision-making techniques that are being leveraged by an ever-growing community of practitioners, including many whose applications have security-sensitive elements. However, while security researchers often utilize such techniques to address problems and AI/ML researchers develop techniques for Big Data analytics applications, neither community devotes much attention to the other. Within security research, AI/ML components are usually regarded as black-box solvers. Conversely, the learning community seldom considers the security/privacy implications entailed in the application of their algorithms when they are designing them. While these two communities generally focus on different directions, where these two fields do meet, interesting problems appear. Researchers working in this intersection have raised many novel questions for both communities and created a new branch of research known as secure learning. The AISec workshop has become the primary venue for this unique fusion of research. In recent years, there has been an increase of activity within the AISec/secure learning community. There are several reasons for this surge. Firstly, machine learning, data mining, and other artificial intelligence technologies play a key role in extracting knowledge, situational awareness, and security intelligence from Big Data. Secondly, companies like Google, Facebook, Amazon, and Splunk are increasingly exploring and deploying learning technologies to address Big Data problems for their customers.

Artificial Intelligence (AI) and Machine Learning (ML) provide a set of useful analytic and decision-making techniques that are being leveraged by an ever-growing community of practitioners, including many whose applications have security-sensitive elements. However, while security researchers often utilize such techniques to address problems and AI/ML researchers develop techniques for Big Data analytics applications, neither community devotes much attention to the other. Within security research, AI/ML components are usually regarded as black-box solvers. Conversely, the learning community seldom considers the security/privacy implications entailed in the application of their algorithms when they are designing them. While these two communities generally focus on different directions, where these two fields do meet, interesting problems appear. Researchers working in this intersection have raised many novel questions for both communities and created a new branch of research known as secure learning. The AISec workshop has become the primary venue for this unique fusion of research. In recent years, there has been an increase of activity within the AISec/secure learning community. There are several reasons for this surge. Firstly, machine learning, data mining, and other artificial intelligence technologies play a key role in extracting knowledge, situational awareness, and security intelligence from Big Data. Secondly, companies like Google, Facebook, Amazon, and Splunk are increasingly exploring and deploying learning technologies to address Big Data problems for their customers.

Passwords are used for user authentication by almost every Internet service today, despite a number of wellknown weaknesses. Numerous attempts to replace passwords have failed, in part because changing users’ behavior has proven to be difficult. One approach to strengthening password-based authentication without changing user experience is to classify login attempts into normal and suspicious activity based on a number of parameters such as source IP, geo-location, browser configuration, and time of day. For the suspicious attempts, the service can then require additional verification, e.g., by an additional phone-based authentication step. Systems working along these principles have been deployed by a number of Internet services but have never been studied publicly. In this work, we perform the first public evaluation of a classification system for user authentication. In particular: (i) We develop a statistical framework for identifying suspicious login attempts. (ii) We develop a fully functional prototype implementation that can be evaluated efficiently on large datasets. (iii) We validate our system on a sample of real-life login data from LinkedIn as well as simulated attacks, and demonstrate that a majority of attacks can be prevented by imposing additional verification steps on only a small fraction of users. (iv) We provide a systematic study of possible attackers against such a system, including attackers targeting the classifier itself.

Fake accounts are a preferred means for malicious users of online social networks to send spam, commit fraud, or otherwise abuse the system. A single malicious actor may create dozens to thousands of fake accounts in order to scale their operation to reach the maximum number of legitimate members. Detecting and taking action on these accounts as quickly as possible is imperative in order to protect legitimate members and maintain the trustworthiness of the network. However, any individual fake account may appear to be legitimate on first inspection, for example by having a real-sounding name or a believable profile. In this work we describe a scalable approach to finding groups of fake accounts registered by the same actor. The main technique is a supervised machine learning pipeline for classifying {\em an entire cluster} of accounts as malicious or legitimate. The key features used in the model are statistics on fields of user-generated text such as name, email address, company or university; these include both frequencies of patterns {\em within} the cluster (e.g., do all of the emails share a common letter/digit pattern) and comparison of text frequencies across the entire user base (e.g., are all of the names rare?). We apply our framework to analyze account data on LinkedIn grouped by registration IP address and registration date. Our model achieved AUC 0.98 on a held-out test set and AUC 0.95 on out-of-sample testing data. The model has been productionalized and has identified more than 250,000 fake accounts since deployment.

Many social networks are predicated on the assumption that a member's online information reflects his or her real identity. In such networks, members who fill their name fields with fictitious identities, company names, phone numbers, or just gibberish are violating the terms of service, polluting search results, and degrading the value of the site to real members. Finding and removing these accounts on the basis of their spammy names can both improve the site experience for real members and prevent further abusive activity. In this paper we describe a set of features that can be used by a Naive Bayes classifier to find accounts whose names do not represent real people. The model can detect both automated and human abusers and can be used at registration time, before other signals such as social graph or clickstream history are present. We use member data from LinkedIn to train and validate our model and to choose parameters. Our best-scoring model achieves AUC 0.85 on a sequestered test set. We ran the algorithm on live LinkedIn data for one month in parallel with our previous name scoring algorithm based on regular expressions. The false positive rate of our new algorithm (3.3%) was less than half that of the previous algorithm (7.0%). When the algorithm is run on email usernames as well as user-entered first and last names, it provides an effective way to catch not only bad human actors but also bots that have poor name and email generation algorithms.

We propose a general framework that converts (ordinary) signature schemes having certain properties into linearly homomorphic signature schemes, i.e., schemes that allow authentication of linear functions on signed data. The security of the homomorphic scheme follows from the same computational assumption as is used to prove security of the underlying signature scheme. We show that the following signature schemes have the required properties and thus give rise to secure homomorphic signatures in the standard model:- The scheme of Waters (Eurocrypt 2005), secure under the computational Diffie-Hellman asumption in bilinear groups.- The scheme of Boneh and Boyen (Eurocrypt 2004, J. Cryptology 2008), secure under the q-strong Diffie-Hellman assumption in bilinear groups.- The scheme of Gennaro, Halevi, and Rabin (Eurocrypt 1999), secure under the strong RSA assumption.- The scheme of Hohenberger and Waters (Crypto 2009), secure under the RSA assumption.Our systems not only allow weaker security assumptions than were previously available for homomorphic signatures in the standard model, but also are secure in a model that allows a stronger adversary than in other proposed schemes.Our framework also leads to efficient linearly homomorphic signatures that are secure against our stronger adversary under weak assumptions (CDH or RSA) in the random oracle model; all previous proofs of security in the random oracle model break down completely when faced with our stronger adversary.

• Gen(): Choose an elliptic curve E/Fp with a point P of prime order n, and an integer s R ← [1, n]. Output pk = (P,Q = [s]P ) and sk = s. • Enc(pk,M): Let pk = (P,Q) and interpret M as a point on E(Fp). Choose r R ← [1, n] and output C = ([r]P,M + [r]Q). • Dec(C, sk). Let sk = s and C = (C1, C2). Output M ′ = C2 − [s]C1. Suppose we have two messagesM,M ′ ∈ E(Fp) and we compute the encryptions C = Enc(pk,M) = ([r]P,M + [r]Q) and C ′ = Enc(pk,M ′) = ([r′]P,M ′ + [r′]Q). Then C +C ′ (added componentwise) is equal to ([r + r′]P, (M + M ′) + [r + r′]Q) — so this is an encryption of M + M ′. Thus given two encryptions of messages, we can obtain an encryption of the sum of the messages without decrypting. (Indeed, the system remains semantically secure.) We say the system is “homomorphic”: encryption computes a group homomorphism from the message space to the ciphertext space. When M and M ′ are points on an elliptic curve, adding the two messages might not make much sense in terms of the underlying meaning of the messages. It would be better if M and M ′ were integers. For example, if M and M ′ represent salaries of Stanford professors, then adding them gives the sum of the salaries (and subsequently dividing by 2 gives the average). So here’s a variant of ElGamal that encrypts integers:

We propose a lattice-based functional encryption scheme for inner product predicates whose security follows from the difficulty of the learning with errors (LWE) problem. This construction allows us to achieve applications such as range and subset queries, polynomial evaluation, and CNF/DNF formulas on encrypted data. Our scheme supports inner products over small fields, in contrast to earlier works based on bilinear maps. Our construction is the first functional encryption scheme based on lattice techniques that goes beyond basic identity-based encryption. The main technique in our scheme is a novel twist to the identity-based encryption scheme of Agrawal, Boneh and Boyen (Eurocrypt 2010). Our scheme is weakly attribute hiding in the standard model.

Deniable encryption, introduced in 1997 by Canetti, Dwork, Naor, and Ostrovsky, guarantees that the sender or the receiver of a secret message is able to "fake" the message encrypted in a specific ciphertext in the presence of a coercing adversary, without the adversary detecting that he was not given the real message. To date, constructions are only known either for weakened variants with separate "honest" and "dishonest" encryption algorithms, or for single-algorithm schemes with non-negligible detection probability. We propose the first sender-deniable public key encryption system with a single encryption algorithm and negligible detection probability. We describe a generic interactive construction based on a public key bit encryption scheme that has certain properties, and we give two examples of encryption schemes with these properties, one based on the quadratic residuosity assumption and the other on trapdoor permutations.

A pairing-friendly curve is a curve over a finite field whose Jacobian has small embedding degree with respect to a large prime-order subgroup. In this paper we construct pairing-friendly genus 2 curves over finite fields Fq whose Jacobians are ordinary and simple, but not absolutely simple. We show that constructing such curves is equivalent to constructing elliptic curves over Fq that become pairing-friendly over a finite extension of Fq. Our main proof technique is Weil restriction of elliptic curves. We describe adaptations of the Cocks–Pinch and Brezing–Weng methods that produce genus 2 curves with the desired properties. Our examples include a parametric family of genus 2 curves whose Jacobians have the smallest recorded ρ-value for simple, non-supersingular abelian surfaces.