Motion capture or mocap systems allow for tracking and recording of human motions at high spatial and temporal resolutions. The resulting 3D mocap data is used for motion analysis in fields such as sports sciences, biomechanics, or computer vision, and in particular for motion synthesis in data-driven computer animation. In view of a rapidly growing corpus of motion data, automatic retrieval, annotation, and classification of such data has become an important research field. Since logically similar motions may exhibit significant spatio-temporal variations, the notion of similarity is of crucial importance in comparing motion data streams. After reviewing various aspects of motion similarity, we discuss as the main contribution of this chapter a relational approach to content-based motion analysis, which exploits the existence of an explicitly given kinematic model underlying the 3D mocap data. Considering suitable combinations of boolean relations between specified body points allows for capturing the motion content while disregarding motion details. Finally, we sketch how such relational features can be used for automatic and efficient segmentation, indexing, retrieval, classification, and annotation of mocap data.

This paper presents new methods for automatic classification and retrieval of motion capture data facilitating the identification of logically related motions scattered in some database. As the main ingredient, we introduce the concept of motion templates (MTs), by which the essence of an entire class of logically related motions can be captured in an explicit and semantically interpretable matrix representation. The key property of MTs is that the variable aspects of a motion class can be automatically masked out in the comparison with unknown motion data. This facilitates robust and efficient motion retrieval even in the presence of large spatio-temporal variations. Furthermore, we describe how to learn an MT for a specific motion class from a given set of training motions. In our extensive experiments, which are based on several hours of motion data, MTs proved to be a powerful concept for motion annotation and retrieval, yielding accurate results even for highly variable motion classes such as cartwheels, lying down, or throwing motions.

Motion capturing has become an important tool in fields such as sports sciences, biometrics, and particularly in computer animation, where large collections of motion material are accumulated in the production process. In order to fully exploit motion databases for reuse and for the synthesis of new motions, one needs efficient retrieval and browsing methods to identify similar motions. So far, only ad-hoc methods for content-based motion retrieval have been proposed, which lack efficiency and rely on quantitative, numerical similarity measures, making it difficult to identify logically related motions. We propose an efficient motion retrieval system based on the query-by-example paradigm, which employs qualitative, geometric similarity measures. This allows for intuitive and interactive browsing in a purely content-based fashion without relying on textual annotations. We have incorporated this technology in a novel user interface facilitating query formulation as well as visualization and ranking of search results.

In this paper we propose a unified approach to efficient indexing and content-based retrieval of human motion capture data as used in data-driven computer animation or computer vision. Opposed to other data formats such as music or video, the kinematic chain (a kind of human skeleton) as underlying model of motion capture data allows to introduce qualitative boolean features describing geometric re lations of specified points of the body. In combination with such geometric features, we introduce the concept of adaptive temporal segmentation of motion data streams, which accounts for the spatio-temporal invariance needed to iden tify logically related motions. This allows us to adopt efficient indexing and fault-tolerant retrieval methods such as fuzzy search. Here, the crucial point is that our adaptive se gmentation not only adjusts to the granularity of the feature function but also to the fuzziness of the query. We present experimental results on a test data set of more than one million frames corresponding to 180 minutes of motion capture data. The linearity of our indexing algorithms guarantees the scalability of our results to much larger data sets.

The reuse of human motion capture data to create new, realistic motions by applying morphing and blending techniques has become an important issue in computer animation. This requires the identification and extraction of logically related motions scattered within some data set. Such content-based retrieval of motion capture data, which is the topic of this paper, constitutes a difficult and time-consuming problem due to significant spatio-temporal variations between logically related motions. In our approach, we introduce various kinds of qualitative features describing geometric relations between specified body points of a pose and show how these features induce a time segmentation of motion capture data streams. By incorporating spatio-temporal invariance into the geometric features and adaptive segments, we are able to adopt efficient indexing methods allowing for flexible and efficient content-based retrieval and browsing in huge motion capture databases. Furthermore, we obtain an efficient preprocessing method substantially accelerating the cost-intensive classical dynamic time warping techniques for the time alignment of logically similar motion data streams. We present experimental results on a test data set of more than one million frames, corresponding to 180 minutes of motion. The linearity of our indexing algorithms guarantees the scalability of our results to much larger data sets.

In this contribution we propose a generic service for realtime access to context-based music information such as lyrics or score data. In our web-based client-server scenario, a client application plays back a particular (waveform) audio recording. During playback, the client connects to a server which in turn identifies the particular piece of audio as well as the current playback position. Subsequently, the server delivers local, i.e., position specific, context-based information on the audio piece to the client. The client then synchronously displays the received information during acoustic playback. We demonstrate how such a service can be established using recent MIR (Music Information Retrieval) techniques such as audio identification and synchronization and present two particular application scenarios.

In the last few years, several algorithms for the automatic alignment of audio and score data corresponding to the same piece of music have been proposed. Among the major drawbacks to these approaches are the long running times as well as the large memory requirements. In this paper we present an algorithm, which solves the synchronization problem accurately and efficiently for complex, polyphonic piano music. In a first step, we extract from the audio data stream a set of highly expressive features encoding note onset candidates separately for all pitches. This makes computations efficient since only a small number of such features is sufficient to solve the synchronization task. Based on a suitable matching model, the best match between the score and the feature parameters is computed by dynamic programming (DP). To further cut down the computational cost in the synchronization process, we introduce the concept of anchor matches, matches which can be easily established. Then the DP-based technique is locally applied between adjacent anchor matches. Evaluation results have been obtained on complex polyphonic piano pieces including Chopin’s Etudes Op. 10.

Preface In the past two decades, motion capture (mocap) systems have been developed that allow to track and record human motions at high spatial and temporal resolutions. The resulting motion capture data is used to analyze human motions in fields such as sports sciences and biometrics (person identification), and to synthesize realistic motion sequences in data-driven computer animation. Such applications require efficient methods and tools for the automatic analysis, synthesis and classification of motion capture data, which constitutes an active research area with many yet unsolved problems. Even though there is a rapidly growing corpus of motion capture data, the academic research community still lacks publicly available motion data, as supplied by [4], that can be freely used for systematic research on motion analysis, synthesis, and classification. Furthermore, a common dataset of annotated and well-documented motion capture data would be extremely valuable to the research community in view of an objective comparison and evaluation of the achieved research results. It is the objective of our motion capture database HDM05 1 to supply free motion capture data for research purposes. HDM05 contains more than tree hours of systematically recorded and well-documented motion capture data in the C3D as well as in the ASF/AMC data format. Furthermore, HDM05 contains for each of roughly 70 motion classes 10 to 50 realizations executed by various actors amounting to roughly 1, 500 motion clips. In this documentation, we give a detailed description of our mocap database HDM05. In Sect. 1, we provide some general information on motion capture data including references to various application fields. A detailed description of the database structure of HDM05 as well as of the content of each mocap file can be found in Sect. 2. We also provide several MATLAB tools comprising a parser for ASF/AMC and C3D as well as visualization, renaming and cutting tools, which are described in Sect. 3. Finally, Sect. 4 summarizes some facts on the mocap file formats ASF/AMC and C3D as used in our database. We appreciate any comments and suggestions for improvement. 1 The motion capture data has been recorded at the Hochschule der Medien (HDM) in the year 2005 under the supervision of Bernhard Eberhardt.