This paper systematically investigates the heuristical optimization technique known as deterministic annealing. This method is applicable to a large class of assignment and partitioning problems. Moreover, the established theoretical results, as well as the general algorithmic solution scheme, are largely independent of the objective functions under consideration. Deterministic annealing is derived from strict minimization principles, including a rigorous convergence analysis. We stress the close relation to homotopy methods, and discuss some of the most important strengths and weaknesses in this framework. Optimization results for unsupervised texture segmentation are presented for an autonomous robotics application.

Cartoon pirate uploading is a very serious problem for the image and video-sharing website. In this study, we propose a new method to detect the characters in 2D-cartoon images, aiming at rejecting pirate uploading automatically. We extract the curve in the cartoon image as the main content and then design a local shape feature named Scalable-Shape Context (SSC) to present the local shape of cartoon. Firstly, we use the Harris-Laplace corner detector to find the key points at multi-scale in the cartoon image, most of which are localized at the junctions of curves. Secondly, the scale of each key point is used as a reference scale for Shape Context (SC) to describe the curvilinear structure around the key points. Then, the matching problem between the key points extracted from the input model and testing image is solved as an optimal assignment problem. Finally, a Hough-voting scheme is employed to find the …

We present a novel approach to measuring similarity between shapes and exploit it for object recognition. In our framework, the measurement of similarity is preceded by: (1) solving for correspondences between points on the two shapes; (2) using the correspondences to estimate an aligning transform. In order to solve the correspondence problem, we attach a descriptor, the shape context, to each point. The shape context at a reference point captures the distribution of the remaining points relative to it, thus offering a globally discriminative characterization. Corresponding points on two similar shapes will have similar shape contexts, enabling us to solve for correspondences as an optimal assignment problem. Given the point correspondences, we estimate the transformation that best aligns the two shapes; regularized thin-plate splines provide a flexible class of transformation maps for this purpose. The dissimilarity between the two shapes is computed as a sum of matching errors between corresponding points, together with a term measuring the magnitude of the aligning transform. We treat recognition in a nearest-neighbor classification framework as the problem of finding the stored prototype shape that is maximally similar to that in the image. Results are presented for silhouettes, trademarks, handwritten digits, and the COIL data set.

Image similarity measures are at the core of every image retrieval system. In this contribution, we provide a systematic overview of distribution-based measures for image similarity. We then empirically compare nine families of color and texture similarity measures summarizing over 1,000 CPU hours of computational experiments. Quantitative performance evaluations are given for classification and image retrieval.Based on the empirical findings a novel image retrieval framework is developed relying on the following fundamental design decisions: First, database items are described by generative probabilistic models. Second, similarity between a query and a database image is measured in terms of how well the corresponding generative model describes or explains the new query. Besides its statistical foundation the proposed procedure has the following key advantages: (i) The probabilistic models can be estimated independently from each other. Thus no joint histogram binning for the complete database as inmost commonly employed methods is necessary. (ii) It is possible to model different cues for different images. (iii) The approach can naturally be extended to more refined models.

We develop an approach to object recognition based on matching shapes and using a resulting measure of similarity in a nearest neighbor classifier. The key algorithmic problem here is that of finding pointwise correspondences between an image shape and a stored prototype shape. We introduce a new shape descriptor, the shape context, which makes this possible, using a simple and robust algorithm. The shape context at a point captures the distribution over relative positions of other shape points and thus summarizes global shape in a rich, local descriptor. We demonstrate that shape contexts greatly simplify recovery of correspondences between points of two given shapes. Once shapes are aligned, shape contexts are used to define a robust score for measuring shape similarity. We have used this score in a nearest-neighbor classifier for recognition of hand written digits as well as 3D objects, using exactly the same distance function. On the benchmark MNIST dataset of handwritten digits, this yields an error rate of 0.63%, outperforming other published techniques.

This paper empirically compares nine image dissimilarity measures that are based on distributions of color and texture features summarizing over 1,000 CPU hours of computational experiments. Ground truth is collected via a novel random sampling scheme for color, and via an image partitioning method for texture. Quantitative performance evaluations are given for classification, image retrieval, and segmentation tasks, and for a wide variety of dissimilarity measures. It is demonstrated how the selection of a measure, based on large scale evaluation, substantially improves the quality of classification, retrieval, and unsupervised segmentation of color and texture images.

In this paper, a systematic optimization approach for clustering proximity or similarity data is developed. Starting from fundamental invariance and robustness properties, a set of axioms is proposed and discussed to distinguish different cluster compactness and separation criteria. The approach covers the case of sparse proximity matrices, and is extended to nested partitionings for hierarchical data clustering. To solve the associated optimization problems, a rigorous mathematical framework for deterministic annealing and mean-field approximation is presented. Efficient optimization heuristics are derived in a canonical way, which also clarifies the relation to stochastic optimization by Gibbs sampling. Similarity-based clustering techniques have a broad range of possible applications in computer vision, pattern recognition, and data analysis. As a major practical application we present a novel approach to the problem of unsupervised texture segmentation, which relies on statistical tests as a measure of homogeneity. The quality of the algorithms is empirically evaluated on a large collection of Brodatz-like micro-texture Mondrians and on a set of real–word images. To demonstrate the broad usefulness of the theory of proximity based clustering the performances of different criteria and algorithms are compared on an information retrieval task for a document database. The superiority of optimization algorithms for clustering is supported by extensive experiments.

Texture is an inherently non-local image property. All common texture descriptors, therefore, have a significant spatial support which renders classical edge detection schemes inadequate for the detection of texture boundaries. In this paper we propose a novel scheme to learn filters for texture edge detection. Textures are defined by the statistical distribution of Gabor filter responses. Optimality criteria for detection reliability and localization accuracy are suggested in the spirit of Canny's edge detector. Texture edges are determined as zero crossings of the difference of the two a posteriori class distributions. An optimization algorithm is designed to determine the best filter kernel according to the underlying quality measure. The effectiveness of the approach is demonstrated on texture mondrians composed from the Brodatz album and a series of synthetic aperture radar (SAR) imagery. Moreover, we indicate how the proposed scheme can be combined with snake-type algorithms for prior-knowledge driven boundary refinement and interactive annotation

of quadruplets (Webpages.url, Webpages..le id, Images .image id, Images.description) will be obtained. For each quadruplet, a triplet (Webpages.url, Web-pages..le id, Images.image id) are extracted and the set of triplets is used by the next step for assembling the nal result. The fourth component of each quadruplet (i.e., Images.description) will be extracted and be used by subquery q3. Subquery q2 is a query against a collection of web pages. This is exactly the kind of query seen in most search engines and in traditional document retrieval systems 24]. The standard approach for evaluating such a query is to use the inverted le index for the document collection 24, 28]. The result of evaluating q2 is a set of pairs (Web-pages..le id, similarity), where similarity > 0 is the similarity of the web page identiied by the le id with the query (the description following \text similar to"). This set of pairs will be used by the next step for assembling the nal result. Subquery q3 is processed after subquery q1 has been processed. A beneet of this evaluation order is that only image descriptions whose corresponding image ids are returned by q1 need to be used to process q3. Note that there is no inverted le index for the descriptions of images. As a result, the similarities between the image descriptions returned by q1 and the image description in the query have to be computed one by one. The evaluation of q3 produces a set of pairs (Images.image id, similarity), where similarity > 0 is the similarity of the description of the image identiied by the image id with the image description in the user query. Again, this set of pairs will be used by the next step for assembling the nal result. As described above, the result of evaluating q1 is a set of triplets (Webpages.url, Webpages..le id, Images .image id), the result of evaluating q2 is a set of pairs (Webpages..le id, similarity), and the result of evaluating q3 is a set of pairs (Images.image id, similarity). We now discuss how to generate the nal result to the user query from these triplets and pairs. The result assembling is accomplished by the following algorithm. 1. Sort the triplet le based on the url eld. Sort the two pair les based on the le id and the image id elds, respectively. 2. For each triplet, say (url 1 , w …

Image quantization and digital halftoning, two fundamental image processing problems, are generally performed sequentially and, in most cases, independent of each other. Color reduction with a pixel-wise defined distortion measure and the halftoning process with its local averaging neighborhood typically optimize different quality criteria or, frequently, follow a heuristic approach without reference to any quantitative quality measure. In this paper, we propose a new model to simultaneously quantize and halftone color images. The method is based on a rigorous cost-function approach which optimizes a quality criterion derived from a simplified model of human perception. It incorporates spatial and contextual information into the quantization and thus overcomes the artificial separation of quantization and halftoning. Optimization is performed by an efficient multiscale procedure which substantially alleviates the computational burden. The quality criterion and the optimization algorithms are evaluated on a representative set of artificial and real-world images showing a significant image quality improvement compared to standard color reduction approaches. Applying the developed cost function, we also suggest a new distortion measure for evaluating the overall quality of color reduction schemes.

Grouping algorithms based on histograms over measured image features have very successfully been applied to textured image segmentation. However, the competing goals of statistical estimation significance demanding few quantization levels versus the necessary richness in representation often prevent a successful application for the color cue, since quantization may result in contouring. We combine a halftoning technique called spatial quantization with distribution-based grouping algorithms to synthesize a powerful color image segmentation technique. The spatial quantization simultaneously determines color palette and halftoning by optimization of a joint cost function. It therefore allows for a highly adapted image representation with a smooth transition of color distributions for non-constant image surfaces.

This paper introduces a novel statistical mixture model for probabilistic grouping of distributional (histogram) data. Adopting the Bayesian framework, we propose to perform annealed maximum a posteriori estimation to compute optimal clustering solutions. In order to accelerate the optimization process, an efficient multiscale formulation is developed. We present a prototypical application of this method for the unsupervised segmentation of textured images based on local distributions of Gabor coefficients. Benchmark results indicate superior performance compared to K-means clustering and proximity-based algorithms.

Dyadzc data refers to a domain with two finite sets of objects in which observations are made for dyads, i.e., pairs with one element from either set. This type of data arises naturally in many application ranging from computational linguistics and information retrieval to preference analysis and computer vision. In this paper, we present a systematic, domain-independent framework of learning from dyadic data by statistical mixture models. Our approach covers different models with fiat and hierarchical latent class structures. We propose an annealed version of the standard EM algorithm for model fitting which is empirically evaluated on a variety of data sets from different domains.

Land usage classification is an essential part of many remote sensing applications for mapping, inventory, and yield estimation. In this contribution, we evaluate the potential of the support vector machines for remote sensing applications. Moreover, we expand this discriminative technique by a novel Bayesian approach to estimate the confidence of each classification. These estimates are combined with a priori knowledge about topological relations of class labels using a contextual classification step based on the iterative conditional mode algorithm (ICM). As shown for Landsat TM imagery, this strategy is highly competitive and outperforms several commonly used classification schemes

We present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a data clustering problem based on sparse proximity data. Dissimilarities of pairs of textured regions are computed from a multiscale Gabor filter image representation. We discuss and compare a class of clustering objective functions which is systematically derived from invariance principles. As a general optimization framework, we propose deterministic annealing based on a mean-field approximation. The canonical way to derive clustering algorithms within this framework as well as an efficient implementation of mean-field annealing and the closely related Gibbs sampler are presented. We apply both annealing variants to Brodatz-like microtexture mixtures and real-word images.

Image quantization and digital halftoning are fundamental problems in computer graphics, which arise when displaying high‐color images on non‐truecolor devices. Both steps are generally performed sequentially and, in most cases, independent of each other. Color quantization with a pixel‐wise defined distortion measure and the dithering process with its local neighborhood optimize different quality criteria or, frequently, follow a heuristic without reference to any quality measure.In this paper we propose a new method to simultaneously quantize and dither color images. The method is based on a rigorous cost‐function approach which optimizes a quality criterion derived from a generic model of human perception. A highly efficient algorithm for optimization based on a multiscale method is developed for the dithered color quantization cost function. The quality criterion and the optimization algorithms are evaluated on a representative set of artificial and real‐world images as well as on a collection of icons. A significant image quality improvement is observed compared to standard color reduction approaches.

Modeling and predicting co-occurrences of events is a fundamental problem of unsupervised learning. We develop a general statistical framework for analyzing co-occurrence data based on probabilistic clustering by mixture models. More specifically, we discuss three models which pursue different modeling goals and which differ in the way they define the probabilistic partitioning of the observations. Adopting the maximum likelihood principle, annealed EM algorithms are derived for parameter estimation. From the class of potential applications in pattern recognition and data analysis, we have chosen document retrieval, language modeling, and unsupervised texture segmentation to test and evaluate the proposed algorithms.