This document describes the technical mechanisms used by censorship regimes around the world to block or impair Internet traffic. It aims to make designers, implementers, and users of Internet protocols aware of the properties being exploited and mechanisms used to censor end- user access to information. This document makes no suggestions on individual protocol considerations, and is purely informational, intended to be a reference.

We present NEL (Network Error Logging), a planet-scale, client-side, network reliability measurement system. NEL is implemented in Google Chrome and has been proposed as a new W3C standard, letting any web site operator collect reports of clients' successful and failed requests to their sites. These reports are similar to web server logs, but include information about failed requests that never reach serving infrastructure. Reports are uploaded via redundant failover paths, reducing the likelihood of shared-fate failures of report uploads. We have designed NEL such that service providers can glean no additional information about users or their behavior compared to what services already have visibility into during normal operation. Since 2014, NEL has been invaluable in monitoring all of Google's domains, allowing us to detect and investigate instances of DNS hijacking, BGP route leaks, protocol deployment bugs, and other problems where packets might never reach our servers. This paper presents the design of NEL, case studies of real outages, and deployment lessons for other operators who choose to use NEL to monitor their traffic.

Despite the pervasive nature of Internet censorship and the continuous evolution of how and where censorship is applied, measurements of censorship remain comparatively sparse. Understanding the scope, scale, and evolution of Internet censorship requires global measurements, performed at regular intervals. Unfortunately, the state of the art relies on techniques that, by and large, require users to directly participate in gathering these measurements, drastically limiting their coverage and inhibiting regular data collection. To facilitate large-scale measurements that can fill this gap in understanding, we develop Iris, a scalable, accurate, and ethical method to measure global manipulation of DNS resolutions. Iris reveals widespread DNS manipulation of many domain names; our findings both confirm anecdotal or limited results from previous work and reveal new patterns in DNS manipulation.

We present techniques for detecting unauthorized DNS root servers in the Internet using primarily endpoint-based measurements from RIPE Atlas, supplemented with BGP routing announcements from RouteViews and RIPE RIS. The first approach analyzes the latency to the root server and the second approach looks for route hijacks. We demonstrate the importance and validity of these techniques by measuring the only root server (“B”) not widely distributed using anycast. Our measurements establish the presence of several DNS proxies and a DNS root mirror.

Understanding censorship requires performing widespread, continuous measurements "on the ground". Yet, measuring censorship is potentially dangerous, due to the threat of retaliation against citizens who perform measurements. We must balance measurement accuracy, reliability, and scalability with user safety which leads us to the question: Can we design censorship measurements that mitigate risk to the users who consent to perform them? Although it is almost certainly impossible to eliminate risk (or even determine if we have succeeded in doing so), we posit that we may be able to reduce risk with measurement techniques that are difficult to observe or distinguish from innocuous network activity. We observe that surveillance and censorship systems have different goals, and thus certain types of measurement techniques may be able to characterize a censorship system without triggering a surveillance system. We design and implement several techniques for measuring censorship that controlled tests suggest might be less risky than existing methods; we also highlight potential pitfalls, limitations, and avenues for future work.

Based on our experiences in measuring censorship in several projects, we frame various ethical questions and challenges that we have encountered. We offer this short document to highlight open questions that we view as important to consider when establishing ethical norms for censorship measurement.

One means of enforcing Web censorship is to return a block page, which informs the user that an attempt to access a webpage is unsuccessful. Detecting block pages can provide a more complete picture of Web censorship, but automatically identifying block pages is difficult because Web content is dynamic, personalized, and may even be in different languages. Previous work has manually detected and identified block pages, which is difficult to reproduce; it is also time-consuming, which makes it difficult to perform continuous, longitudinal studies of censorship. This paper presents an automated method both to detect block pages and to fingerprint the filtering products that generate them. Our automated method enables continuous measurements of block pages; we found that our methods successfully detect 95% of block pages and identify five filtering tools, including a tool that had not been previously identified "in the wild".

Censorship circumvention systems that use HTTP as cover traffic make tradeoffs between deniability and performance by offering either deniability at the expense of performance (e.g., Infranet) or performance at the expense of deniability (e.g., StegoTorus). These systems do so because HTTP is typically very asymmetric, with very little capacity to carry covert data in each HTTP GET request; higher throughput channels achieve performance by generating sequences of HTTP GET requests that do not mimic normal user traffic patterns. Fortunately, the emergence of new web services makes it increasingly common for any individual HTTP GET requests to contain more entropy. For example, site-specific search services create GET requests that contain sequences of search terms that can encode more bits than a single deniable HTTP request otherwise would. In this paper, we design a new encoding technique that uses web search terms to encode hidden messages in an upstream channel for censorship circumvention; implement the encoding technique in a system that resists fingerprinting attacks; and compare the security and performance of Facade to existing censorship circumvention systems that use HTTP as cover traffic.