Learning Hybrid Models for Image Annotation with Partially Labeled Data

Extensive labeled data for image annotation systems, which learn to assign class labels to image regions, is difficult to obtain. We explore a h ybrid model frame- work for utilizing partially labeled data that integrates a generative topic model for image appearance with discriminative label prediction. We propose three al- ternative formulations for imposing a spatial smoothness prior on the image la- bels. Tests of the new models and some baseline approaches on three real image datasets demonstrate the effectiveness of incorporating t he latent structure.