Machine learning develops intelligent computer systems that are able to generalize from previously seen examples. A new domain of machine learning, in which the prediction must satisfy the additional constraints found in structured data, poses one of machine learning's greatest challenges: learning functional dependencies between arbitrary input and output domains. This volume presents and analyzes the state of the art in machine learning algorithms and theory in this novel field. The contributors discuss applications as diverse as machine translation, document markup, computational biology, and information extraction, among others, providing a timely overview of an exciting field. Contributors Yasemin Altun, Gi??i??khan Bakir [no dot over i], Olivier Bousquet, Sumit Chopra, Corinna Cortes, Hal Daumi??i?? III, Ofer Dekel, Zoubin Ghahramani, Raia Hadsell, Thomas Hofmann, Fu Jie Huang, Yann LeCun, Tobias Mann, Daniel Marcu, David McAllester, Mehryar Mohri, William Stafford Noble, Fernando Pi??i??rez-Cruz, Massimiliano Pontil, Marc'Aurelio Ranzato, Juho Rousu, Craig Saunders, Bernhard Schi??i??lkopf, Matthias W. Seeger, Shai Shalev-Shwartz, John Shawe-Taylor, Yoram Singer, Alexander J. Smola, Sandor Szedmak, Ben Taskar, Ioannis Tsochantaridis, S.V.N Vishwanathan, Jason Weston Gi??i??khan Bakir [no dot over i] is Research Scientist at the Max Planck Institute for Biological Cybernetics in Ti??i??bingen, Germany. Thomas Hofmann is a Director of Engineering at Google's Engineering Center in Zurich and Adjunct Associate Professor of Computer Science at Brown University. Bernhard Schi??i??lkopf is Director of the Max Planck Institute for Biological Cybernetics and Professor at the Technical University Berlin. Alexander J. Smola is Senior Principal Researcher and Mac hine Learning Program Leader at National ICT Australia/Australian National University, Canberra. Ben Taskar is Assistant Professor in the Computer and Information Science Department at the University of Pennsylvania. S. V. N. Vishwanathan is Senior Researcher in the Statistical Machine Learning Program, National ICT Australia with an adjunct appointment at the Research School for Information Sciences and Engineering, Australian National University.

Many kernel learning algorithms, including support vector machines, result in a kernel machine, such as a kernel classifier, whose key component is a weight vector in a feature space implicitly introduced by a positive definite kernel function. This weight vector is usually obtained by solving a convex optimization problem. Based on this fact we present a direct method to build sparse kernel learning algorithms by adding one more constraint to the original convex optimization problem, such that the sparseness of the resulting kernel machine is explicitly controlled while at the same time performance is kept as high as possible. A gradient based approach is provided to solve this modified optimization problem. Applying this method to the support vectom machine results in a concrete algorithm for building sparse large margin classifiers. These classifiers essentially find a discriminating subspace that can be spanned by a small number of vectors, and in this subspace, the different classes of data are linearly well separated. Experimental results over several classification benchmarks demonstrate the effectiveness of our approach.

Energy-Based Models (EBMs) capture dependencies between variables by as- sociating a scalar energy to each configuration of the variab les. Inference consists in clamping the value of observed variables and finding config urations of the re- maining variables that minimize the energy. Learning consists in finding an energy function in which observed configurations of the variables a re given lower energies than unobserved ones. The EBM approach provides a common theoretical frame- work for many learning models, including traditional discr iminative and genera- tive approaches, as well as graph-transformer networks, co nditional random fields, maximum margin Markov networks, and several manifold learning methods. Probabilistic models must be properly normalized, which sometimes requires evaluating intractable integrals over the space of all poss ible variable configura- tions. Since EBMs have no requirement for proper normalization, this problem is naturally circumvented. EBMs can be viewed as a form of non-probabilistic factor graphs, and they provide considerably more flexibility in th e design of architec- tures and training criteria than probabilistic approaches .