In computer vision research, shape is usually referred to the geometric properties of objects and it is very important in detecting and recognizing objects. Shape representation, the data structure to describe the shape, is the basis for shape analysis. Graph, due to its power in representing various relationship among shape parts, is frequently used in describing part-based 2D shapes. The flexibility of graphs, however, results in difficulties in matching and recognition of graphic shapes, especially when the number of edges is large because the matching of the edges usually involves the matching of the two nodes it connects. In many graphical shape representation methods, however, the number of edges will grow quadratically with respect to the number of nodes. That is partly the reason why recent research on shape analysis often deal with graphs with a limited number of nodes, or with edges abandoned. In order to keep more shape details to improve the performance of shape recognition, graphs with larger number of nodes should be applied. This motivates us to design a shape representation method that can keep the connectivity linear to the number of nodes while most of the shape details are kept. For images where edges can be reasonably detected, we are arguing that images' intensity edges are the right connectivity information that should be kept. A new shape representation, multi-sequence shape (or MSS), is proposed. The matching, classification and indexing algorithms of MSS are also proposed and experiments are conducted to demonstrate the advantage of our shape representation method. In our representation method, the number of edges of graphs is linear to the number of nodes. The time complexity of our matching algorithms are O(N2 ) where N is the number of nodes. We will also show that our representation method is suitable for indexing so that retrieving a shape from a 60,000 shape database only takes a few milliseconds using a pre-computed index.Our method could match and retrieve shapes very fast and accurately, based on three contributions of this dissertation. We proposed an efficient partial matching algorithm to find the most similar parts of two sequential shapes, which is the base of our proposed shape representation. Our algorithm does not need to exhaustively search all possible pairs of subsequences. Instead, we use a dynamic programming algorithm (due to Smith Waterman) to find the most similar parts efficiently The complexity of our method to find similar parts of two contours of length m and n, is only O(m · n). We compared our matching algorithm with other methods using contour shape models. Empirical analysis shows that our approach is about 10 to 20 times faster in rotation-invariant recognition because our method does not need to search all possible rotations to find the optimal orientation. Because our matching algorithm only extract similar parts of two contours, shape variance, such as occlusion, can be better tolerated and hence our algorithm is more robust to shape variance. In contrast to arbitrary distance functions that are used by previous methods, we use a probabilistic similarity measurement, p-value, to evaluate the similarity of two shapes. We conducted experiments on several public shape databases and the result indicates that our method outperforms state-of-the-art global and partial shape matching algorithms.Based on the partial sequence shape matching algorithm, we propose a new shape representation, the multi-sequence shape, to represent shapes in both binary images and realistic images. We propose a matching algorithm for multi-sequence shapes and an algorithm that could recognize multi-sequence shapes from cluttered background as well. We are arguing that the proposed shape representation method, compared to traditional graphical shape models, such as contour model, sparse graph model, and dense graph model, is straightforward in nature and less complicated in representation and matching whereas contains sufficient shape details for common shape recognition tasks.A structured learning algorithm is also proposed to improve the shape classification. Traditional methods for shape classification involve the establishment of point correspondences between shapes to produce matching scores, which are in turn used as similarity measures for classification. Learning techniques have been applied only in the second stage of this process, after the matching scores have been obtained. We take a different approach by learning point-to-point matching measures to produce similarity scores that minimize the classification loss. Instead of simply taking for granted the scores obtained by matching and then learning a classifier, we embed both matching and classification together within single machine learning scheme that optimizes the shape classification accuracy. The solution is based on a max-margin formulation in the structured prediction setting. Experiments in several shape databases reveal that such integrated learning algorithm substantially improves the classification accuracy of existing methods.

User clicks on a URL in response to a query are extremely useful predictors of the URL's relevance to that query. Exact match click features tend to suffer from severe data sparsity issues in web ranking. Such sparsity is particularly pronounced for new URLs or long queries where each distinct query-url pair will rarely occur. To remedy this, we present a set of straightforward yet informative query-url n -gram features that allows for generalization of limited user click data to large amounts of unseen query-url pairs. The method is motivated by techniques leveraged in the NLP community for dealing with unseen words. We find that there are interesting regularities across queries and their preferred destination URLs; for example, queries containing "form" tend to lead to clicks on URLs containing "pdf". We evaluate our set of new query-url features on a web search ranking task and obtain improvements that are statistically significant at a p -value < 0.0001 level over a strong baseline with exact match clickthrough features.

Many traditional methods for shape classification involve establishing point correspondences between shapes to produce matching scores, which are in turn used as similarity measures for classification. Learning techniques have been applied only in the second stage of this process, after the matching scores have been obtained. In this paper; instead of simply taking for granted the scores obtained by matching and then learning a classifier, we learn the matching scores themselves so as to produce shape similarity scores that minimize the classification loss. The solution is based on a max-margin formulation in the structured prediction setting. Experiments in shape databases reveal that such an integrated learning algorithm substantially improves on existing methods.

Moving object classification in far-field video is a key component of smart surveillance systems. In this paper, we propose a reliable system for person-vehicle classification which works well in challenging real-word conditions, including the presence of shadows, low resolution imagery, perspective distortions, arbitrary camera viewpoints, and groups of people. Our system runsin real-time (30 Hz) on conventional machines and has low memory consumption. We achieved accurate results by relying on powerful discriminative features, including a novel measure of object deformation based on differences of histograms of oriented gradients. We also provide an interactive user interface, enabling users to specify regions of interest for each class and correct for perspective distortions by specifying different sizes indifferent positions of the camera view. Finally, we use anautomatic adaptation process to continuously update the parameters of the system so that its performance increases for a particular environment. Experimental results demonstrate the effectiveness of our system in standard dataset and a variety of video clips captured with our surveillance cameras.

This paper presents an efficient partial shape matching method based on the Smith-Waterman algorithm. For two contours of m and n points respectively, the complexity of our method to find similar parts is only O(mn). In addition to this improvement in efficiency, we also obtain comparable accurate matching with fewer shape descriptors. Also, in contrast to arbitrary distance functions that are used by previous methods, we use a probabilistic similarity measurement, p-value, to evaluate the similarity of two shapes. Our experiments on several public shape databases indicate that our method outperforms state-of-the-art global and partial shape matching algorithms in various scenarios.

Traditional stereo matching algorithms are limited in their ability to produce accurate results near depth discontinuities, due to partial occlusions and violation of smoothness constraints. In this paper, we use small baseline multiflash illumination to produce a rich set of feature maps that enable the acquisition of discontinuity preserving point correspondences. First, from a single multiflash camera, we formulate a qualitative depth map using a gradient domain method that encodes object relative distances. Then, in a multiview setup, we exploit shadows created by light sources to compute an occlusion map. Finally, we demonstrate the usefulness of these feature maps by incorporating them into two different dense stereo correspondence algorithms, the first based on local search and the second based on belief propagation. Experimental results show that our enhanced stereo algorithms are able to extract high-quality discontinuity preserving correspondence maps from scenes that are extremely challenging for conventional stereo methods. We also demonstrate that small baseline illumination can be useful to handle specular reflections in stereo imagery. Different from most existing active illumination techniques, our method is simple, inexpensive, and compact and requires no calibration of light sources.

Photo album management, an application of content-based image browsing/retrieval, has attracted much attention in recent years. Identities of individuals appearing in photos are the most important aspect of photo browsing. However, face recognition generally does not work effectively in such situations due to the large variations in pose, illumination and sometimes poor quality of images. We present a system for browsing photo albums. The system automatically detects and stores information about the locations of faces in the photos. A similarity function based on face arrangement is defined. Photos are then clustered based on the similarity function using a proposed clustering algorithm. The system also represents photos of an event using a composite image. The composite image is built from representative faces and an image that represents the event. Experiments indicate that the face arrangement features are effective in representing the semantic content of the photos and are appropriate for photo albums.

We describe a novel approach to appearance-based hand pose estimation which relies on multiple cameras to improve accuracy and resolve ambiguities caused by selfocclusions. Rather than estimating 3D geometry as most previous multi-view imaging systems, our approach uses multiple views to extend current exemplar-based methods that estimate hand pose by matching a probe image with a large discrete set of labeled hand pose images. We formulate the problem in a MAP (maximum a posteriori) framework, where the information from multiple cameras is fused to provide reliable hand pose estimation. Our quantitative experimental results show that correct estimation rate is much higher using our multi-view approach than using a single-view approach.

Currently, sharp discontinuities in depth and partial occlusions in multiview imaging systems pose serious challenges for many dense correspondence algorithms. However, it is important for 3D reconstruction methods to preserve depth edges as they correspond to important shape features like silhouettes which are critical for understanding the structure of a scene. In this paper, we show how active illumination algorithms can produce a rich set of feature maps that are useful in dense 3D reconstruction. We start by showing a method to compute a qualitative depth map from a single camera, which encodes object relative distances and can be used as a prior for stereo. In a multiview setup, we show that along with depth edges, binocular half-occluded pixels can also be explicitly and reliably labeled. To demonstrate the usefulness of these feature maps, we show how they can be used in two different algorithms for dense stereo correspondence. Our experimental results show that our enhanced stereo algorithms are able to extract high quality, discontinuity preserving correspondence maps from scenes that are extremely challenging for conventional stereo methods.

A robust pose estimation approach is proposed by combining facial appearance asymmetry and 3D geometry in a coarse-to-fine framework. The rough face pose is first estimated by analyzing the asymmetry of the distribution of the facial component detection confidences on an image, which actually implies an intrinsic relation between the face pose and the facial appearance. Then, this rough face pose, as well as error bandwidth, is utilized into a 3D-to-2D geometrical model matching to refine the pose estimation. The proposed approach is able to track a face with fast motion in front of cluttered background and recover its pose robustly and accurately in real- time. Experiment results are provided to demonstrate its efficiency and accuracy.

A hierarchical automatic classification system of the image database has been realized, which concerns the following semantic information: Day, Night, Sunrise/Sunset, Indoor, Outdoor, Building and Landscape. Several discriminative features are selected through analyzing the image contents and comparing the present low-level features. The application of Support Vector Machines(SVMs)in the hierarchical semantic image classification is investigated. To improve the accuracy,a reject scheme is adopted in our system through revising the standard SVM output into the probabilistic output. The system also achieves good performance on testing sample from the Google retrieval result,so it can be thought a good filter for the image retrieval system.

This paper presents a method to automatically locate facial feature points under large variations in pose, illumination and facial expressions. First we propose a method to calculate probabilistic-like output for each pixel of image. This probabilistic-like output describes the possibility of the pixel to be the center of specified object. A Gaussian Mixture Model is used to approximate the distribution of probabilistic-like output. The centers of these Gaussians are assigned with a probabilistic-like measure and they are considered as candidate feature points. There might be one or more candidate feature points in each facial region. A 3D model of facial feature points is built to enforce constraints on the localization results of feature points. Compared with Active Shape Model (ASM) and its variant methods, our method could accommodate larger variations in pose, lighting and face expressions. Moreover, it is less sensitive to initialization errors, accurate, and fast. It takes a computer with P4 CPU about 10ms to locate the five feature points (two eye centers, two mouth corners and nose tip). The feature localization accuracy is comparable with the accuracy of manually labeled features and it is robust to noise (glasses, beards). Experiments on FERET gallery and PIE are reported in this paper as well.

In this paper, we propose a framework to semi-automatically annotate faces in family photo albums. The core of the framework is the features used to define face similarity and this results in the learning algorithm used to refine automatic face annotation. We have adopted similarity based search and relevance feedback ideas developed for content-based image retrieval and a set of simple yet effective color and texture based features, in addition to the traditional face recognition features, in performing candidate annotation search. The experimental evaluation of the proposed approach has been conducted with a family album of 1707 photos and the results show that the proposed approach is an effective and efficient one for semi-automatic family photo album annotation.

Automatic annotation of photographs is one of the most desirable needs in family photograph management systems. In this paper, we present a learning framework to automate the face annotation in family photograph albums. Firstly, methodologies of content-based image retrieval and face recognition are seamlessly integrated to achieve automated annotation. Secondly, face annotation is formulated in a Bayesian framework, in which the face similarity measure is defined as maximum a posteriori (MAP) estimation. Thirdly, to deal with the missing features, marginal probability is used so that samples which have missing features are compared with those having the full feature set to ensure a non-biased decision. The experimental evaluation has been conducted within a family album of few thousands of photographs and the results show that the proposed approach is effective and efficient in automated face annotation in family albums.

Here, we propose a new learning strategy for head pose estimation. Our approach uses nonlinear interpolation to estimate the head pose using the learning result from face images of two head poses. Advantage of our method to regression method is that it only requires training images of two head poses and better generalization ability. It outperforms existed methods, such as regression and multiclass classification method, on both synthesis and real face images. Average head pose estimation error of yaw rotation is about 4/sup 0/, which proves that our method is effective in head pose estimation.

Automatic annotation of photographs is one of the most desirable needs in family photograph management systems. This paper presents a new framework for semiautomated face annotation in family photo album applications. Based on the Bayesian similarity measure, which integrates face recognition and content-based image retrieval, the system is able to provide both face annotation and similar face retrieval, and thus significantly improve the management of family albums.

The image semantic concept is very important and useful for the image retrieval and browsing. The semantic concept of the image can be inferred from low-level features such as color, shape, texture, etc. In this paper, we propose an approach for the building semantic extraction of the scene image using SVM. We select the edge direction histogram and Gabor texture as the discriminative features to realize the image semantic extraction. Experiments have been done by using the standard two-class SVM and one-class SVM and the results obtained are presented. By comparing the experimental results, we conclude that the two-class SVM yields better performance than the one-class SVM. However, the benefit of using one-class SVM is due to its time saving in training. This classifier does not need many versatile negative examples and achieves a high classification accuracy.