Addressing the Space Overhead of Vector Quotient Filter

Filters, also known as Approximate Membership Query data structures, show whether an item is present in a dataset. Filters are widely used in storage systems to avoid unnecessary I/O operations. In distributed database systems, effective memory management can be achieved by utilizing filters to reduce the I/O requests of non-volatile memory. Vector Quotient Filter is a filter that achieves high performance in uniform datasets. However, it has limitations in terms of space and performance when applied to skewed datasets. It does not count duplicate items and gives them equal space, leading to large space overhead. We examine how Vector Quotient Filter can overcome this space overhead issue in skewed datasets by applying a counting algorithm.