Surface electromyogram (sEMG) signals are widely employed as a neural control source for lower-limb exoskeletons, in which gait recognition based on sEMG is particularly important. Many scholars have taken measures to improve the accuracy of gait recognition, but several real-time limitations affect its applicability, of which variation in the load styles is obvious. The purposes of this study are to (1) investigate the impact of different load styles on gait recognition; (2) study whether good gait recognition performance can be obtained when a convolutional neural network (CNN) is used to deal with the sEMG image from sparse multichannel sEMG (SMC-sEMG); and (3) explore whether the control system of the lower-limb exoskeleton trained by sEMG from part of the load styles still works efficiently in a real-time environment where multiload styles are required. In addition, we discuss an effective method to improve gait recognition at the levels of the load styles. In our experiment, fifteen able-bodied male graduate students with load (20% of body weight) and using three load styles (SBP = backpack, SCS = cross shoulder, SSS = straight shoulder) were asked to walk uniformly on a treadmill. Each subject performed 50 continuous gait cycles under three speeds (V3 = 3 km/h, V5 = 5 km/h, and V7 = 7 km/h). A CNN was employed to deal with sEMG images from sEMG signals for gait recognition, and back propagation neural networks (BPNNs) and support vector machines (SVMs) were used for comparison by dealing with the same sEMG signal. The results indicated that (1) different load styles had remarkable impact on the gait recognition at three speeds under three load styles (p < 0.001); (2) the performance of gait recognition from the CNN was better than that from the SVM and BPNN at each speed (84.83%, 81.63%, and 83.76% at V3; 93.40%, 88.48%, and 92.36% at V5; and 90.1%, 86.32%, and 85.42% at V7, respectively); and (3) when all the data from three load styles were pooled as testing sets at each speed, more load styles were included in the training set, better performance was obtained, and the statistical analysis suggested that the kinds of load styles included in training set had a significant effect on gait recognition (p = 0.002), from which it can be concluded that the control system of a lower-limb exoskeleton trained by sEMG using only some load styles is not sufficient in a real-time environment.

As surface electromyogram (sEMG) signals have the ability to detect human movement intention, they are commonly used to be control inputs. However, gait sub-phase classification typically requires monotonous manual labeling process, and commercial sEMG acquisition devices are quite bulky and expensive, thus current sEMG-based gait sub-phase recognition systems are complex and have poor portability. This study presents a low-cost but effective end-to-end sEMG-based gait sub-phase recognition system, which contains a wireless multi-channel signal acquisition device simultaneously collecting sEMG of thigh muscles and plantar pressure signals, and a novel neural network-based sEMG signal classifier combining long-short term memory (LSTM) with multilayer perceptron (MLP). We evaluated the system with subjects walking under five conditions: flat terrain at 5 km/h, flat terrain at 3 km/h, 20 kg backpack at 5 km/h, 20 kg shoulder bag at 5 km/h and 15° slope at 5 km/h. Experimental results show that the proposed method achieved average classification accuracies of 94.10%, 87.25%, 90.71%, 94.02%, and 87.87%, respectively, which were significantly higher than existing recognition methods. Additionally, the proposed system had a good real-time performance with low average inference time in the range of 3.25 ~ 3.31 ms.

Using video sequences to restore 3D human poses is of great significance in the field of motion capture. This paper proposes a novel approach to estimate 3D human action via end-to-end learning of deep convolutional neural network to calculate the parameters of the parameterized skinned multi-person linear model. The method is divided into two main stages: (1) 3D human pose estimation based on a single frame image. We use 2D/3D skeleton point constraints, human height constraints, and generative adversarial network constraints to obtain a more accurate human-body model. The model is pre-trained using open-source human pose datasets; (2) Human-body pose generation based on video streams. Combined with the correlation of video sequences, a 3D human pose recovery method based on video streams is proposed, which uses the correlation between videos to generate a smoother 3D pose. In addition, we compared the proposed 3D human pose recovery method with the commercial motion capture platform to prove the effectiveness of the proposed method. To make a contrast, we first built a motion capture platform through two Kinect (V2) devices and iPi Soft series software to obtain depth-camera video sequences and monocular-camera video sequences respectively. Then we defined several different tasks, including the speed of the movements, the position of the subject, the orientation of the subject, and the complexity of the movements. Experimental results show that our low-cost method based on RGB video data can achieve similar results to commercial motion capture platform with RGB-D video data.

As lower-limb exoskeleton and prostheses are developed to become smarter and to deploy man–machine collaboration, accurate gait recognition is crucial, as it contributes to the realization of real-time control. Many researchers choose surface electromyogram (sEMG) signals to recognize the gait and control the lower-limb exoskeleton (or prostheses). However, several factors still affect its applicability, of which variation in the loads is an essential one. This study aims to (1) investigate the effect of load variation on gait recognition; and to (2) discuss whether a lower-limb exoskeleton control system trained by sEMG from different loads works well in multi-load applications. In our experiment, 10 male college students were selected to walk on a treadmill at three different speeds (V3 = 3 km/h, V5 = 5 km/h, and V7 = 7 km/h) with four different loads (L0 = 0, L20 = 20%, L30 = 30%, L40 = 40% of body weight, respectively), and 50 gait cycles were performed. Back propagation neural networks (BPNNs) were used for gait recognition, and a support vector machine (SVM) and k-nearest neighbor (k-NN) were used for comparison. The result showed that (1) load variation has significant effects on the accuracy of gait recognition (p < 0.05) under the three speeds when the loads range in L0, L20, L30, or L40, but no significant impact is found when the loads range in L0, L20, or L30. The least significant difference (LSD) post hoc, which can explore all possible pair-wise comparisons of means that comprise a factor using the equivalent of multiple t-tests, reveals that there is a significant difference between the L40 load and the other three loads (L0, L20, L30), but no significant difference was found among the L0, L20, and L30 loads. The total mean accuracy of gait recognition of the intra-loads and inter-loads was 91.81%, and 69.42%, respectively. (2) When the training data was taken from more types of loads, a higher accuracy in gait recognition was obtained at each speed, and the statistical analysis shows that there was a substantial influence for the kinds of loads in the training set on the gait recognition accuracy (p < 0.001). It can be concluded that an exoskeleton (or prosthesis) control system that is trained in a single load or the parts of loads is insufficient in the face of multi-load applications.

In this paper, we propose a novel gait representation based on 3D-CNN, i.e., learning spatio-temporal multi-scale gait identity features (GaitID) using the 3-dimensional convolutional networks. Our contributions include: 1) explore different numbers of input frames for 3D-CNN model, 2) evaluate different features and gait representations in 3D-CNN, and 3) improve the net structure to learn multi-scale gait features with low dimensions. Nearest neighbor (NN) classifier was applied to identify the gait. When compared with other existing methods, the results reported on the CASIA-B dataset demonstrated that the proposed method not only achieved a competitive performance, but also still retained the discriminative power in a very low dimension (128-D), even with a simpler classifier.

Recently, skeleton-based human action recognition has been receiving significant attention from various research communities due to its robustness, succinctness, and view-invariant representation. Most of the existing skeleton-based methods use either well-designed classifiers with hand-crafted features or current neural network (RNN) to recognize human actions. In this paper, inspired by the deep convolutional neural network's breakthroughs in the image domain, we transform a skeleton sequence into an image and perform end-to-end learning of deep convolutional neural network (CNN). The skeleton sequence based image contains spatial temporal information. Our proposed method is tested on the NTU RGB+D dataset which is so far the largest skeleton-based human action dataset, and achieves the state-of-the-art performance for both the cross-view and cross-subject evaluations.

Human gait, as a soft biometric, helps to recognize people by walking. To further improve the recognition performances, we propose a novel video sensor-based gait representation, DeepGait, using deep convolutional features. DeepGait is generated by using an pre-trained VGG-D net without any fine-tuning. When compared with other traditional hand-crafted gait representations (eg. GEI, FDF and GFI etc.) experimentally on OU-ISR large population (OULP) dataset and CASIA-B dataset, DeepGait has been shown that the performances of the proposed method is outstanding under different walking variations (view, clothing, carrying bags). The OULP dataset, which includes 4007 subjects, makes our result reliable in a statically way. Even in a very low dimension, our proposed gait representation still outperforms the commonly used 11264-dimensional GEI. For further comparison, all the gait representation vectors are available.