Sketch-based 3D shape retrieval has become an important research topic in content-based 3D object retrieval. The aim of this track is to measure and compare the performance of sketch-based 3D shape retrieval methods implemented by different participants over the world. The track is based on a new sketch-based 3D shape benchmark, which contains two types of sketch queries and two versions of target 3D models. In this track, 7 runs have been submitted by 5 groups and their retrieval accuracies were evaluated using 7 commonly used retrieval performance metrics. We hope that the benchmark, its corresponding evaluation code, and the comparative evaluation results of the state-of-the-art sketch-based 3D model retrieval algorithms will contribute to the progress of this research direction for the 3D model retrieval community.

We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an existing data base of objects. The sketches turn out to be generally quite abstract with large local and global deviations from the original shape. Based on this observation, we decide to use a bag-of-features approach over computer generated line drawings of the objects. We develop a targeted feature transform based on Gabor filters for this system. We can show objectively that this transform is better suited than other approaches from the literature developed for similar tasks. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, our approach is significantly better than any other system described so far.

Since prehistoric times, humans have used sketching to depict our visual world. Even today, sketching is possibly the only rendering technique readily available to all humans. To understand how humans sketch objects, we perform two experiments. In the first experiment, we analyze the distribution of non-expert sketches of everyday objects such as ‘teapot’ or ‘car’. We ask participants to sketch objects of a given category and gather 20,000 unique sketches evenly distributed over 250 object categories. The second experiment targets 3d shape retrieval, and we gather 1,814 sketches that are related to the categories in an existing dataset of 3d shapes. The sketches in both datasets turn out to be generally quite abstract with large local and global deviations from the original shape. Based on the first sketch dataset, we perform a perceptual study and find that humans can correctly identify the object category of a sketch 73% of the time. We develop a targeted feature transform for sketches that is based on a bagof-features approach, yields a compact representation and comes with suitable invariance properties. Using this representation, we develop the first computational recognition method for classifying human object sketches. We compare human performance against the computational model for which we use multi-class support vector machines, trained on the sketch dataset, to classify sketches. The resulting recognition method is able to identify unknown sketches with 56% accuracy (chance is 0.4%). Using the computational model, we demonstrate an interactive sketch recognition system. Based on the second dataset, we develop a system for 3d object retrieval using sketched feature lines as input. The system employs a view-based approach, matching the input against computer generated line drawings of the objects, using the bag-of-features representation developed earlier. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, we show objectively that our approach performs significantly better than any other system described so far.

Humans have used sketching to depict our visual world since prehistoric times. Even today, sketching is possibly the only rendering technique readily available to all humans. This paper is the first large scale exploration of human sketches. We analyze the distribution of non-expert sketches of everyday objects such as 'teapot' or 'car'. We ask humans to sketch objects of a given category and gather 20,000 unique sketches evenly distributed over 250 object categories. With this dataset we perform a perceptual study and find that humans can correctly identify the object category of a sketch 73% of the time. We compare human performance against computational recognition methods. We develop a bag-of-features sketch representation and use multi-class support vector machines, trained on our sketch dataset, to classify sketches. The resulting recognition method is able to identify unknown sketches with 56% accuracy (chance is 0.4%). Based on the computational model, we demonstrate an interactive sketch recognition system. We release the complete crowd-sourced dataset of sketches to the community.

Photosketcher is an interactive system for progressively synthesizing novel images using only sparse user sketches as input. Photosketcher works on the image content exclusively; it doesn't require keywords or other metadata associated with the images. Users sketch the rough shape of a desired image part, and Photosketcher searches a large collection of images for it. The search is based on a bag-of-features approach that uses local descriptors for translation-invariant retrieval of image parts. Composition is based on user scribbles: from the scribbles, Photosketcher predicts the desired part using Gaussian mixture models and computes an optimal seam using graph cuts. To further reduce visible seams, users can blend the composite image in the gradient domain.

We present ongoing work on object category recognition from binary human outline sketches. We first define a novel set of 187 "sketchable" object categories by extracting the labels of the most frequent objects in the LabelMe dataset. In a large-scale experiment, we then gather a dataset of over 5,500 human sketches, evenly distributed over all categories. We show that by training multi-class support vector machines on this dataset, we can classify novel sketches with high accuracy. We demonstrate this in an inter-active sketching application that progressively updates its category prediction as users add more strokes to a sketch.

We introduce a benchmark for evaluating the performance of large-scale sketch-based image retrieval systems. The necessary data are acquired in a controlled user study where subjects rate how well given sketch/image pairs match. We suggest how to use the data for evaluating the performance of sketch-based image retrieval systems. The benchmark data as well as the large image database are made publicly available for further studies of this type. Furthermore, we develop new descriptors based on the bag-of-features approach and use the benchmark to demonstrate that they significantly outperform other descriptors in the literature.

Manually locating an image in a large collection has become infeasible with the recent rapid growth in size of such collections. Nowadays, even private collections easily contain tens of thousands of images; public collections have long passed the billion images mark. Current approaches for finding images in large collections, therefore, try to confine the set of images by returning only those images that correspond to certain properties defined by a query. Such properties can include: keywords, semantic information associated with the images, similarity to an example image, a rough sketch of the desired outlines, or any combination thereof.

We address the problem of fast, large scale sketch-based image retrieval, searching in a database of over one million images. We show that current retrieval methods do not scale well towards large databases in the context of interactively supervised search and propose two different approaches for which we objectively evaluate that they significantly outperform existing approaches. The proposed descriptors are constructed such that both the full color image and the sketch undergo exactly the same preprocessing steps. We first search for an image with similar structure, analyzing gradient orientations. Then, best matching images are clustered based on dominant color distributions, to offset the lack of color-based decision during the initial search. Overall, the query results demonstrate that the system offers intuitive access to large image databases using a user-friendly sketch-and-browse interface.

We introduce a system for progressively creating images through a simple sketching and compositing interface. A large database of over 1.5 million images is searched for matches to a user's binary outline sketch; the results of this search can be combined interactively to synthesize the desired image. We introduce image descriptors for the task of estimating the difference between images and binary outline sketches. The compositing part is based on graph cut and Poisson blending. We demonstrate that the resulting system allows generating complex images in an intuitive way.

We present a new, efficient and easy to use collision detection scheme for real-time collision detection between highly deformable tetrahedral models. Tetrahedral models are a common representation of volumetric meshes which are often used in physically based simulations, e.g. in Virtual surgery. In a deformable models environment collision detection usually is a performance bottleneck since the data structures used for efficient intersection tests need to be rebuilt or modified frequently. Our approach minimizes the time needed for building a collision detection data structure. We employ an infinite hierarchical spatial grid in which for each single tetrahedron in the scene a well fitting grid cell size is computed. A hash function is used to project occupied grid cells into a finite ID hash table. Only primitives mapped to the same hash index indicate a possible collision and need to be checked for intersections. This results in a high performance collision detection algorithm which does not depend on user defined parameters and thus flexibly adapts to any scene setup.

We present an image editing tool that allows to deform and composite image regions using an intuitive sketch-based interface. Users simply draw the outline of the source image region and sketch a new boundary shape onto the location where this region is to be pasted. The deformation optimizes a shape distortion energy, and we use Poisson cloning for subsequent compositing. Since the correspondence between the source and target boundary curves is not known a priori, we propose an alternating optimization process that interleaves minimization of the deformation energy w.r.t. the image region interior and the mapping between the boundary curves, thus automatically determining the boundary correspondence and deforming the image at the same time. Thanks to the particular design of the deformation energy, its gradient can be efficiently computed, making the entire image editing framework interactive and convenient for practical use.

Soft tissue simulation is a challenging task in computer graphics. As a part of a virtual surgery application, physically plausible deformations of soft tissue have to be computed in real-time. Challenges include realistic but fast simulation of soft tissue as well as collision detection and collision response. The aim of this thesis has been to create a framework that tackles those challenges. Therefore, existing approaches of soft tissue simulation, collision detection and collision response have been discussed and their advantages and drawbacks have been analyzed. Next, new approaches have been proposed that overcome the discussed limitations. Finally, all suggested algorithms have been assembled into a framework that allows to accurately simulate deformable soft tissue models consisting of several thousand tetrahedrons and their collisions in real-time at interacti ve frame rates. A soft tissue simulator capable of simulating models consisting of several thousands of tetrahedrons in real-time has been implemented and successfully tested. Sev– III –