In interdisciplinary spaces such as digital health, datasets that are complex to collect, require specialist facilities, and/or are collected with specific populations have value in a range of different sectors. In this study we collected a simulated free-living dataset, in a smart home, with 12 participants (six people with Parkinson's, six carers). We explored their initial perceptions of the sensors through interviews and then conducted two data exploration workshops, wherein we showed participants the collected data and discussed their views on how this data, and other data relating to their Parkinson's symptoms, might be shared across different sectors. We provide recommendations around how participants might be better engaged in considering data sharing in the early stages of research, and guidance for how research might be configured to allow for more informed data sharing practices in the future.

IntroductionTurning in gait digital parameters may be useful in measuring disease progression in Parkinson's disease (PD), however challenges remain over algorithm validation in real-world settings. The influence of clinician observation on turning outcomes is poorly understood. Our objective is to describe a unique in-home video dataset and explore the use of turning parameters as biomarkers in PD.Methods11 participants with PD, 11 control participants stayed in a home-like setting living freely for 5 days (with two sessions of clinical assessment), during which high-resolution video was captured. Clinicians watched the videos, identified turns and documented turning parameters.ResultsFrom 85 hours of video 3869 turns were evaluated, averaging at 22.7 turns per hour per person. 6 participants had significantly different numbers of turning steps and/or turn duration between "ON" and "OFF" medication states. Positive Spearman correlations were seen between the Movement Disorders Society-sponsored revision of the Unified Parkinson's Disease Rating Scale III score with a) number of turning steps (rho = 0.893, p < 0.001), and b) duration of turn (rho = 0.744, p = 0.009) "OFF" medications. A positive correlation was seen "ON" medications between number of turning steps and clinical rating scale score (rho = 0.618, p = 0.048). Both cohorts took more steps and shorter durations of turn during observed clinical assessments than when free-living.ConclusionThis study shows proof of concept that real-world free-living turn duration and number of turning steps recorded can distinguish between PD medication states and correlate with gold-standard clinical rating scale scores. It illustrates a methodology for ecological validation of real-world digital outcomes.

Parkinson's disease (PD) is a chronic neurodegenerative condition that affects a patient's everyday life. Authors have proposed that a machine learning and sensor-based approach that continuously monitors patients in naturalistic settings can provide constant evaluation of PD and objectively analyse its progression. In this paper, we make progress toward such PD evaluation by presenting a multimodal deep learning approach for discriminating between people with PD and without PD. Specifically, our proposed architecture, named MCPD-Net, uses two data modalities, acquired from vision and accelerometer sensors in a home environment to train variational autoencoder (VAE) models. These are modality-specific VAEs that predict effective representations of human movements to be fused and given to a classification module. During our end-to-end training, we minimise the difference between the latent spaces corresponding to the two data modalities. This makes our method capable of dealing with missing modalities during inference. We show that our proposed multimodal method outperforms unimodal and other multimodal approaches by an average increase in F1-score of 0.25 and 0.09, respectively, on a data set with real patients. We also show that our method still outperforms other approaches by an average increase in F1-score of 0.17 when a modality is missing during inference, demonstrating the benefit of training on multiple modalities.

This paper looks to explore the challenges faced when producing a set of annotations from videos produced by a pilot study evaluating 24 participants (12 with Parkinson's disease, each accompanied by a healthy volunteer control participant) who are free-living in a house embedded with a platform of sensors. We discuss the outcome measures chosen to annotate from the videos and the controlled vocabularies formulated for this task, the tools and processes, how we intend to achieve standardisation and normalisation of the annotations, and how to improve quality and re-usability of the annotation dataset.

Introduction The impact of disease-modifying agents on disease progression in Parkinson’s disease is largely assessed in clinical trials using clinical rating scales. These scales have drawbacks in terms of their ability to capture the fluctuating nature of symptoms while living in a naturalistic environment. The SPHERE (Sensor Platform for HEalthcare in a Residential Environment) project has designed a multi-sensor platform with multimodal devices designed to allow continuous, relatively inexpensive, unobtrusive sensing of motor, non-motor and activities of daily living metrics in a home or a home-like environment. The aim of this study is to evaluate how the SPHERE technology can measure aspects of Parkinson’s disease.Methods and analysis This is a small-scale feasibility and acceptability study during which 12 pairs of participants (comprising a person with Parkinson’s and a healthy control participant) will stay and live freely for 5 days in a home-like environment embedded with SPHERE technology including environmental, appliance monitoring, wrist-worn accelerometry and camera sensors. These data will be collected alongside clinical rating scales, participant diary entries and expert clinician annotations of colour video images. Machine learning will be used to look for a signal to discriminate between Parkinson’s disease and control, and between Parkinson’s disease symptoms ‘on’ and ‘off’ medications. Additional outcome measures including bradykinesia, activity level, sleep parameters and some activities of daily living will be explored. Acceptability of the technology will be evaluated qualitatively using semi-structured interviews.Ethics and dissemination Ethical approval has been given to commence this study; the results will be disseminated as widely as appropriate.

Monitoring the progression of an action towards completion offers fine grained insight into the actor's behaviour. In this work, we target detecting the completion moment of actions, that is the moment when the action's goal has been successfully accomplished. This has potential applications from surveillance to assistive living and human-robot interactions. Previous effort required human annotations of the completion moment for training (i.e. full supervision). In this work, we present an approach for moment detection from weak video-level labels. Given both complete and incomplete sequences, of the same action, we learn temporal attention, along with accumulated completion prediction from all frames in the sequence. We also demonstrate how the approach can be used when completion moment supervision is available. We evaluate and compare our approach on actions from three datasets, namely HMDB, UCF101 and RGBD-AC, and show that temporal attention improves detection in both weakly-supervised and fully-supervised settings.

We introduce completion moment detection for actions - the problem of locating the moment of completion, when the action's goal is confidently considered achieved. The paper proposes a joint classification-regression recurrent model that predicts completion from a given frame, and then integrates frame-level contributions to detect sequence-level completion moment. We introduce a recurrent voting node that predicts the frame's relative position of the completion moment by either classification or regression. The method is also capable of detecting incompletion. For example, the method is capable of detecting a missed ball-catch, as well as the moment at which the ball is safely caught. We test the method on 16 actions from three public datasets, covering sports as well as daily actions. Results show that when combining contributions from frames prior to the completion moment as well as frames post completion, the completion moment is detected within one second in 89% of all tested sequences.

Action completion detection is the problem of modelling the action's progression towards localising the moment of completion - when the action's goal is confidently considered achieved. In this work, we assess the ability of two temporal models, namely Hidden Markov Models (HMM) and Long-Short Term Memory (LSTM), to localise completion for six object interactions: switch, plug, open, pull, pick and drink. We use a supervised approach, where annotations of pre-completion and post-completion frames are available per action, and fine-tuned CNN features are used to train temporal models. Tested on the Action-Completion-2016 dataset, we detect completion within 10 frames of annotations for ~75% of completed action sequences using both temporal models. Results show that fine-tuned CNN features outperform hand-crafted features for localisation, and that observing incomplete instances is necessary when incomplete sequences are also present in the test set.

An action is completed when its goal has been successfully achieved. Using current state-of-the-art depth features, designed primarily for action recognition, an incomplete sequence may still be classified as its complete counterpart due to the overlap in evidence. In this work we show that while features can perform comparably for action recognition, they vary in their ability to recognise incompletion. Experimenting on a novel dataset of 414 complete/incomplete object interaction sequences, spanning six actions and captured using an RGB-D camera, we test for completion using binary classification on labelled data. Results show that by selecting the suitable feature per action, we achieve 95.7% accuracy for recognising action completion.

Today, using medical imaging devices is essential for disease diagnosis and medical researches. Among these devices, Magnetic Resonance Imaging has the main role. Segmentation of these images is more difficult than natural images because their functional sensitivity is higher than other images. Up to now, many different algorithms have been suggested for segmentation of this type of images. In this paper, we propose an approach in order to improve ant colony algorithm efficiency. In this approach, ant's direction and its tendency to go to the next site is regarded for calculating the probability of choosing the next site by the ant. Moreover, in calculating the probability of the ant's next move, we try to make a balance between the effect of the ant direction and the amount of pheromone distributed. Then this algorithm is used for segmentation of brain magnetic resonance images and diagnosing tumors.

In this paper we describe designing and implementation of a powerful, fast and compact simple 3D modeler (SM3D). In addition to saving cost and time (due to high processing speed), 3D objects can be created with minimum system resources by using this application. Easy learning and using are other strengths of this application. Modularity using classification and applying Dynamic-Link Library files are noted aspects that are regarded in writing the source code and this causes separation of main part and user interface, so the application can be easily expanded in the future. Ability to create primary objects and also applying advanced transformations and modifiers have been considered. Moreover, ability to select points of an object and move them is another prominent feature. Working with the camera, its settings and creating desired viewpoints are other professional features. Also, saving and loading object's information from a file and export objects to other popular types of files are included in this application.

In this paper, we present an interactive algorithm to separate foreground and background regions of natural images (natural image matting) using ant colony optimization. Today, image matting is one of the most challenging and interesting research fields in image processing. In our approach instead of preparing a trimap, the user specifies foreground and background regions by some red and blue scribbles. Then by minimizing local energy function of all pixels alpha matte is estimated. Our approach not only needs a little interaction with the user but also by applying ant colony algorithm on color images, finds homogenous regions of the image and yields good results compared with other methods. In other words, the local energy of a pixel is obtained using traveled path by the pixel ant and since the ant tends to move to pixels similar to beginning pixel, homogenous regions of the image are detected. Moreover, we use some techniques like vectorization in the implementation of our algorithm in order to decrease time complexity. Experimental results show our algorithm advantages.