A Secure Approach for Human Computer Interaction Using Human Hand Action

Hand actions classification is an imperative field for acquiring smart functionality in modern electronic devices because hand actions classification offers interactive and innovative methods to communicate and interact. Therefore, we develop a novel architecture based on you only looking at coefficients (YOLACT), a real-time instance segmentation approach, and a temporal relation network (TRN) for hand actions understanding. In addition, our framework consists of a face recognition-based security network (FRB-SN) for user identification. We trained the YOLACT and the TRN models using the segmented version of the 20BN jester dataset composed of hand actions images and ground truths while the FRB-SN is trained using the VGGFace2 dataset. For testing, the YOLACT is used to segment the object from the given image sequence and then passed to the TRN-trained model to predict the corresponding action. Our experimental results showed that the accuracy and frame rate of the proposed framework are competitive.