AFQN: approximate Q n estimation in data streams

We present afqn (Approximate Fast Q n ), a novel algorithm for approximate computation of the Q n scale estimator in a streaming setting, in the sliding window model. It is well-known that computing the Q n estimator exactly may be too costly for some applications, and the problem is a fortiori exacerbated in the streaming setting, in which the time available to process incoming data stream items is short. In this paper we show how to efficiently and accurately approximate the Q n estimator. As an application, we show the use of afqn for fast detection of outliers in data streams. In particular, the outliers are detected in the sliding window model, with a simple check based on the Q n scale estimator. Extensive experimental results on synthetic and real datasets confirm the validity of our approach by showing up to three times faster updates per second. Our contributions are the following ones: (i) to the best of our knowledge, we present the first approximation algorithm for online computation of the Q n scale estimator in a streaming setting and in the sliding window model; (ii) we show how to take advantage of our UDDSketch algorithm for quantile estimation in order to quickly compute the Q n scale estimator; (iii) as an example of a possible application of the Q n scale estimator, we discuss how to detect outliers in an input data stream.