An Investigation of Hierarchical Bit Vectors

Hierarchical bit vectors represent sets of integers using a collection of bit vectors. At the top level, a single bit is set iff the set is non-empty. The bits of this next level summarize ranges of the elements. In the case of a binary hierarchical bit vector the two bits of the next level summarize two ranges: the lower half and the upper half of the possible elements. At the lowest level each bit records the membership of a particular integer. Hierarchical bit vectors find application in information re trieval, bioinformatics, non- averaging sets, and the conversion of NFAs to DFAs. Competing data structures for such applications include simple bit vectors and tree-based str uctures such as skip-lists. A com- parison of hierarchical bit vectors with two other represen tations (simple bit vectors and binary search trees) is presented. The comparison includes both analytical and empirical analysis of the hierarchical bit vectors. The analytical results show that as the size of the set used in creases, hierarchical bit vec- tors enjoy an advantage over tree-based structures and that as the sets used become sparser, hierarchical bit vectors perform better than standard bit v ectors. The empirical results con- firm that hierarchical bit vectors sit in between the other tw o. The empirical investigation also highlights the impact that the processor cache has on the break-even points.