In this work, we propose a novel data-driven approach to recover missing or corrupted motion capture data, either in the form of 3D skeleton joints or 3D marker trajectories. We construct a knowledge-base that contains prior existing knowledge, which helps us to make it possible to infer missing or corrupted information of the motion capture data. We then build a kd-tree in parallel fashion on the GPU for fast search and retrieval of this already available knowledge in the form of nearest neighbors from the knowledge-base efficiently. We exploit the concept of histograms to organize the data and use an off-the-shelf radix sort algorithm to sort the keys within a single processor of GPU. We query the motion missing joints or markers, and as a result, we fetch a fixed number of nearest neighbors for the given input query motion. We employ an objective function with multiple error terms that substantially recover 3D joints or marker trajectories in parallel on the GPU. We perform comprehensive experiments to evaluate our approach quantitatively and qualitatively on publicly available motion capture datasets, namely CMU and HDM05. From the results, it is observed that the recovery of boxing, jumptwist, run, martial arts, salsa, and acrobatic motion sequences works best, while the recovery of motion sequences of kicking and jumping results in slightly larger errors. However, on average, our approach executes outstanding results. Generally, our approach outperforms all the competing state-of-the-art methods in the most test cases with different action sequences and executes reliable results with minimal errors and without any user interaction.

Lower back pain is experienced by approximately 70% of the world's population, contributing to the worldwide burden of disease. Americans experience lower back pain at an estimated economic cost of $560–$635 billion. One contributing factor for back pain is posture, and more particularly, lack of awareness of dysfunctional posture. For example, many people sit in a slouched or forward-bent position, exacerbated by poor ergonomics while sitting or extended use of handheld digital devices while standing, such as looking down at a smartphone for long periods of time. This report describes a stacksitting technique that is one of the components of the Gokhale Method® for healthy, relaxed sitting and increased awareness of posture. The stacksitting process is illustrated with a case study, where the shape of the spine and the muscle activity are analyzed in parallel during three sitting styles: slouching, arched, and stacksitting. The spine curvature was characterized by the Gokhale SpineTracker™ wearable, which consists of five sensor units attached along the subjects' spines that are used to plot the spinal curve on a digital device such as a smartphone, tablet, or computer. Surface electromyographic (SEMG) recordings were made from the right upper trapezius, left upper trapezius, right midback, and left midback with a second device (Myoscan Pro sensors recorded with Biograph Procomp Infinity) while participants were seated in three postures: a slouched (forward-bent) position, an upright arched position, and an upright stacksitting position as trained by a Gokhale Method teacher. The case observations showed no significant difference in trapezius SEMG activity during each of the three positions. There was a slight increase in SEMG activity of the midback during stacksitting (1.1 μV) as compared with when slouched (0.64 μV), and a significant increase in SEMG activity when sitting arched (4.9 μV). As expected, the spinal activity tracking device showed significant straightening of the lower spine during the stacksitting position as compared with the slouched and arched positions. The observations suggest that the stacksitting position can be taught to others in a way that allows the vertebrae to be parallel to each other with very low levels of corresponding muscle activity. In contrast, sitting in an arched or slouched position could increase asymmetrical pressure on the disks, contributing to vertebral wedging, which could also contribute to spinal disk bulging and eventual back injury. The observations suggest that proper coaching may foster a stacksitting position of the spine, which could foster a healthier posture than slouched or arched spinal positions.

Introduction: Freezing of gait (FoG) is one of the most disabling gait disorders in Parkinson's disease (PD), reflecting motor and cognitive impairments, mainly related to dopamine deficiency. Recent studies investigating kinematic and kinetic factors affecting gait in these patients showed a postural instability characterized by disturbed weight-shifting, inappropriate anticipatory postural adjustment, worse reactive postural control, and a difficulty executing complex motor tasks (i.e. sit-to-walk). These symptoms are difficult to alleviate and not very responsive to Levodopa. For this reason, additional therapeutic actions based on specific therapeutic protocols may help patients with their daily lives. We conducted a randomized control trial aimed to test if two clinical protocols for PD patients with FoG were effective to improve postural control. Methods: Rehabilitation protocols, conceived to improve gait, were based on learning motor exercises with the Action Observation plus Sonification (AOS) technique, or by the use of external sensory cues. We collected biomechanical data (Center of Mass COM, Center of Pressure COP, and moving timings), using the sit-to-walk task as a measure of motor and gait performance. Results: Kinetic and kinematic data showed that when treatment effects consolidate, patients treated with AOS protocol are more efficient in merging subsequent motor tasks (sit-to-stand and gait initiation), and diminished the total moving time and the area of the COP positions. Conclusion: We demonstrated for the first time that PD patients with FoG treated with an AOS protocol aimed at relearning appropriate gait patterns increased balance control and re-acquired more efficient postural control.

Lower back pain is one of the most prevalent diseases in Western societies. A large percentage of European and American populations suffer from back pain at some point in their lives. One successful approach to address lower back pain is postural training, which can be supported by wearable devices, providing real-time feedback about the user's posture. In this work, we analyze the changes in posture induced by postural training. To this end, we compare snapshots before and after training, as measured by the Gokhale SpineTracker™. Considering pairs of before and after snapshots in different positions (standing, sitting, and bending), we introduce a feature space, that allows for unsupervised clustering. We show that resulting clusters represent certain groups of postural changes, which are meaningful to professional posture trainers.

Lower back pain is one of the leading causes for musculoskeletal disability throughout the world. A large percentage of the population suffers from lower back pain at some point in their life. One non-invasive approach to reduce back pain is postural modification which can be learned through training. In this context, wearables are becoming more and more prominent since they are capable of providing feedback about the user's posture in real-time. Optimal, healthy posture depends on the position (sitting, standing, hinging) the user is in. Meaningful feedback needs to adapt to the current position and, in the best case, identify the position automatically to minimize necessary interactions from the user. In this work, we present results of classifying the positions of users based on the readings of the device. We computed various features and evaluated the performance of K-Nearest Neighbors, Extra Trees, Artificial Neural Networks and AdaBoost for global inter-subject classification as well as for personalized subject-specific classification.

Freezing of gait (FoG) is a disabling symptom associated with falls, with little or no responsiveness to pharmacological treatment. Current protocols used for rehabilitation are based on the use of external sensory cues. However, cued strategies might generate an important dependence on the environment. Teaching motor strategies without cues [i.e., action observation (AO) plus Sonification] could represent an alternative/innovative approach to rehabilitation that matters most on appropriate allocation of attention and lightening cognitive load. We aimed to test the effects of a novel experimental protocol to treat patients with Parkinson’s disease (PD) and FoG, using functional, and clinical scales. The experimental protocol was based on AO plus Sonification. 12 patients were treated with 8 motor gestures. They watched eight videos showing an actor performing the same eight gestures, and then tried to repeat each gesture. Each video was composed by images and sounds of the gestures. By means of the Sonification technique, the sounds of gestures were obtained by transforming kinematic data (velocity) recorded during gesture execution, into pitch variations. The same 8 motor gestures were also used in a second group of 10 patients; which were treated with a standard protocol based on a common sensory stimulation method. All patients were tested with functional and clinical scales before, after, at 1 month, and 3 months after the treatment. Data showed that the experimental protocol have positive effects on functional and clinical tests. In comparison with the baseline evaluations, significant performance improvements were seen in the NFOG questionnaire, and the UPDRS (parts II and III). Importantly, all these improvements were consistently observed at the end, 1 month, and 3 months after treatment. No improvement effects were found in the group of patients treated with the standard protocol. These data suggest that a multisensory approach based on AO plus Sonification, with the two stimuli semantically related, could help PD patients with FoG to relearn gait movements, to reduce freezing episodes, and that these effects could be prolonged over time.

One major challenge for 3D pose estimation from a single RGB image is the acquisition of sufficient training data. In particular, collecting large amounts of training data that contain unconstrained images and are annotated with accurate 3D poses is infeasible. We therefore propose to use two independent training sources. The first source consists of images with annotated 2D poses and the second source consists of accurate 3D motion capture data. To integrate both sources, we propose a dual-source approach that combines 2D pose estimation with efficient and robust 3D pose retrieval. In our experiments, we show that our approach achieves state-of-the-art results and is even competitive when the skeleton structure of the two sources differ substantially.

In western societies a huge percentage of the population suffers from some kind of back pain at least once in their life. There are several approaches addressing back pain by postural modifications. Postural training and activity can be tracked by various wearable devices most of which are based on accelerometers. We present research on the accuracy of accelerometer-based posture measurements. To this end, we took simultaneous recordings using an optical motion capture system and a system consisting of five accelerometers in three different settings: On a test robot, in a template, and on actual human backs. We compare the accelerometer-based spine curve reconstruction against the motion capture data. Results show that tilt values from the accelerometers are captured highly accurate, and the spine curve reconstruction works well.

In this work we address the challenging problem of 3D human pose estimation from single images. Recent approaches learn deep neural networks to regress 3D pose directly from images. One major challenge for such methods, however, is the collection of training data. Specifically, collecting large amounts of training data containing unconstrained images annotated with accurate 3D poses is infeasible. We therefore propose to use two independent training sources. The first source consists of accurate 3D motion capture data, and the second source consists of unconstrained images with annotated 2D poses. To integrate both sources, we propose a dual-source approach that combines 2D pose estimation with efficient 3D pose retrieval. To this end, we first convert the motion capture data into a normalized 2D pose space, and separately learn a 2D pose estimation model from the image data. During inference, we estimate the 2D pose and efficiently retrieve the nearest 3D poses. We then jointly estimate a mapping from the 3D pose space to the image and reconstruct the 3D pose. We provide a comprehensive evaluation of the proposed method and experimentally demonstrate the effectiveness of our approach, even when the skeleton structures of the two sources differ substantially.

The characterization and abstraction of large multivariate time series data often poses challenges with respect to effectiveness or efficiency. Using the example of human motion capture data challenges exist in creating compact solutions that still reflect semantics and kinematics in a meaningful way. We present a visual-interactive approach for the semi-supervised labeling of human motion capture data. Users are enabled to assign labels to the data which can subsequently be used to represent the multivariate time series as sequences of motion classes. The approach combines multiple views supporting the user in the visualinteractive labeling process. Visual guidance concepts further ease the labeling process by propagating the results of supportive algorithmic models. The abstraction of motion capture data to sequences of event intervals allows overview and detail-on-demand visualizations even for large and heterogeneous data collections. The guided selection of candidate data for the extension and improvement of the labeling closes the feedback loop of the semisupervised workflow. We demonstrate the effectiveness and the efficiency of the approach in two usage scenarios, taking visualinteractive learning and human motion synthesis as examples.

Motion capture data have been widely used in applications ranging from video games and animations to simulations and virtual environments. Moreover, all data-driven approaches for analysis and synthesis of motions are depending on motion capture data. Although multiple large motion capture datasets are freely available for research, there is no system which can provide a centralised access to all of them in an organised manner. In this paper, we show that using a relational database management system RDBMS to store data not only provide such a centralised access to the data, but also allows to include other sensor modalities e.g., accelerometer data and various semantic annotations. We present two applications for our system: a motion capture player where motions sequences can be retrieved from large datasets using SQL queries and the automatic construction of statistical models which can further be used for complex motion analysis and motions synthesis tasks.

This paper introduces a novel and efficient segmentation method designed for articulated hand motion. The method is based on a graph representation of temporal structures in human hand-object interaction. Along with the method for temporal segmentation we provide an extensive new database of hand motions. The experiments performed on this dataset show that our method is capable of a fully automatic hand motion segmentation which largely coincides with human user annotations.

The work at hand outlines a recording setup for capturing hand and finger movements of musicians. The focus is on a series of baseline experiments on the detectability of coloured markers under different lighting conditions. With the goal of capturing and recording hand and finger movements of musicians in mind, requirements for such a system and existing approaches are analysed and compared. The results of the experiments and the analysis of related work show that the envisioned setup is suited for the expected scenario.

Digital media content comes in a wide variety of modalities and representations. Although they have obvious semantic and structural difference, many of them can be unwrapped into a one-dimensional parameter domain, e.g., time, one spatial dimension. Novel content can then be generated in this parameter domain by computing sequences of elements that are optimal according to an objective to be minimized and in addition satisfy a number of user-defined constraints. Examples for this type of content generation task are audio synthesis, human motion synthesis or architectural texture synthesis. In that work, we present a generalized algorithm for this type of content generation task. We demonstrate the potential of our technique on a selection of content creation tasks, namely the generation of extended animation sequences from motion capture libraries and the example-based synthesis of architectural geometry such as buildings and street blocks.

Motion capture data have been widely used in applications ranging from video games and animations to simulations and virtual environments. Moreover, all data-driven approaches for analysis and synthesis of motions are depending on motion capture data. Although multiple large motion capture data sets are freely available for research, there is no system which can provide a centralized access to all of them in an organized manner. In this paper we show that using a relational database management system (RDBMS) to store data does not only provide such a centralized access to the data, but also allows to include other sensor modalities (e.g. accelerometer data) and various semantic annotations. We present two applications for our system: A motion capture player where motions sequences can be retrieved from large datasets using SQL queries and the automatic construction of statistical models which can further be used for complex motion analysis and motions synthesis tasks.

Different techniques have been developed for capturing and retrieval, action recognition and video based reconstruction of human motion data in the past years. In this paper, we focus on how these techniques can be adapted to handle quadruped motion capture data and which new applications may appear. We discuss some particularities that must be considered during large animal motion capture. For retrieval, we derive suitable feature sets from quadrupeds motion capture data to perform fast searches for similar motions. Based on the retrieval techniques, the action recognition can be performed on the input motion capture sequences as well as on input video streams. We further present a data-driven approach to reconstruct quadruped motions from video data.

One major challenge for 3D pose estimation from a single RGB image is the acquisition of sufficient training data. In particular, collecting large amounts of training data that contain unconstrained images and are annotated with accurate 3D poses is infeasible. We therefore propose to use two independent training sources. The first source consists of images with annotated 2D poses and the second source consists of accurate 3D motion capture data. To integrate both sources, we propose a dual-source approach that combines 2D pose estimation with efficient and robust 3D pose retrieval. In our experiments, we show that our approach achieves state-of-the-art results when both sources are from the same dataset, but it also achieves competitive results when the motion capture data is taken from a different dataset.

The work at hand presents a method to assess the quality of human sleep within a non-laboratory environment. The monitoring of patients is performed by means of a Kinect device. This results in a non-invasive method which is independent of immediate physical contact to subjects. The results of a study which was carried out as proof of concept are discussed and compared with the polysomnography-based gold standard of sleep analysis. When medical data are concerned, confidentiality is always an issue. This is no less important when monitoring people in their own homes, especially when they are in a situation as vulnerable as sleep. To meet the upcoming challenge of protecting people's privacy while still offering analyses of their data we introduce a blurring method to the acquired data and evaluate the use of our sleep detection test on such de-identified data sets.