Disentangling High-Level Factors and Their Features with Conditional Vector Quantized VAEs

•We propose a disentangled representation that models the labels and their features.•Our proposal includes a novel dependency structure and a two-step learning procedure.•It allows accurate control of labels and their features in the generated images.•We show several examples of label and feature manipulation for 2D images.•Our approach improves disentanglement properties and the quality of generated images.