Energy-Based Models in Document Recognition and Computer Vision

The Machine Learning and Pattern Recognition commu- nities are facing two challenges: solving thenormalization problem, and solving the deep learning problem. The normalization problem is related to the difficulty of training probabilistic models over large spaces while keep - ing them properly normalized. In recent years, the ML and Natural Language communities have devoted considerable efforts to circumventing this problem by developing "un- normalized" learning models for tasks in which the output is highly structured (e.g. English sentences). This class of models was in fact originally developed during the 90's in the handwriting recognition community, and includes Graph Transformer Networks, Conditional Random Fields, Hidden Markov SVMs, and Maximum Margin Markov Net- works. We describe these models within the unifying frame- work of "Energy-Based Models" (EBM). The Deep Learning Problem is related to the issue of training all the levels of a recognition system (e.g. seg- mentation, feature extraction, recognition, etc) in an int e- grated fashion. We first consider "traditional" methods for deep learning, such as convolutional networks and back- propagation, and show that, although they produce very low error rates for handwriting and object recognition, they re - quire many training samples. We show that using unsuper- vised learning to initialize the layers of a deep network dra - matically reduces the required number of training samples, particularly for such tasks as the recognition of everyday objects at the category level.